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GUIDELINES  AND  TOOLS  FOR  VBS2  MISSION  AFTER  ACTION  REVIEWS: 
DEVELOPMENT  AND  EVALUATION 

EXECUTIVE  SUMMARY 


Research  Requirement: 

As  Army  trainers  embrace  desktop  simulations  such  as  the  Virtual  Battlespace-2  (VBS2) 
game,  the  need  has  emerged  for  a  tool  to  help  after  action  review  (AAR)  facilitators  put  the 
game’s  new  simulation-based  AAR  capabilities  into  practice.  The  need  exists  in  the  larger 
context  of  developing  and  communicating  tactics,  techniques,  and  procedures  (TTP)  to  keep  pace 
with  the  rapidly  changing  operational  environment.  To  meet  the  need,  the  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  initiated  research  to  develop  tailored  AAR 
guidelines  and  tools  for  trainers  of  VBS2  missions.  The  research  built  on  earlier  work  that 
produced  a  Soldier ’s  Toolbox  for  Developing  TTP,  equipping  units  with  a  Soldier-friendly 
package  to  help  them  capture  and  share  their  hard-won  combat  lessons. 

Procedure: 

A  user-friendly  VBS2  AAR  Guide  was  created  on  the  basis  of  a  literature  review  and 
observations  of  AARs  in  three  separate  Army  courses.  Formative  evaluation  employed  a  multi¬ 
stage,  multi-source  approach.  The  Guide  was  initially  developed  by  subject  matter  experts 
(SME)  and  behavioral  researchers  through  a  draft-review-revise  process.  During  AAR  sessions, 
the  Guide  was  subsequently  used  by  course  facilitators,  who  then  provided  feedback.  Based 
upon  the  feedback  received,  the  Guide  was  refined  and  the  final  product  was  transitioned  to  the 
target  audience. 

In  concert  with  the  Guide’s  development,  exploratory  research  was  conducted  to  assess 
instructor  and  student  perceptions  of  the  effects  of  AARs  with  VBS2  on  learning  and  training 
outcomes.  After  executing  a  mission  using  VBS2,  experimental  groups  participated  in  AARs 
that  employed  VBS2  AAR  capabilities.  Control  groups  participated  in  AARs  that  used  no  VBS2 
capabilities.  Facilitators  in  the  experimental  condition  used  the  Guide  to  structure  and  conduct 
AARs,  while  control  condition  facilitators  did  not.  Both  qualitative  and  quantitative  data  were 
collected  using  observation  protocols,  surveys,  and  “after  action”  discussions  and  evaluations  of 
performance. 

Findings: 

Observational  data  and  feedback  indicated  that  the  AAR  functions  of  VBS2  are  currently 
being  underutilized  by  facilitators.  Feedback  from  participants  was  positive  for  all  aspects  of  the 
Guide.  Instructors  indicated  the  Guide’s  contents  were  sufficient  and  clear,  without  being 
burdensome.  Instructors’  feedback  suggests  the  Guide  is  a  useful  resource  and  adds  substantive 
value  during  the  AARs  of  VBS2  exercises. 

Strong  support  was  found  for  the  positive  impact  of  utilizing  the  Guide  during  VBS2 
AARs.  Every  instructional  aspect  of  the  AAR  that  was  measured,  and  several  aspects  of 
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simulation-based  training  were  rated  more  positively  by  students  in  the  experimental  condition 
than  the  control  condition.  Observational  data  indicated  a  wider  range  of  instructor  and  student 
activities  during  AARs  with  VBS2  than  those  without.  Observational  data  also  indicated  greater 
student  participation  in  AARs  with  VBS2.  Though  limited,  instructor  data  was  positive 
regarding  AARs  with  VBS2.  Finally,  employment  of  the  VBS2  AAR  capabilities  significantly 
increased  “buy-in”  for  simulation-based  training  among  Soldiers. 

Utilization  and  Dissemination  of  Findings: 

The  Guide  is  a  useful  tool  to  help  trainers  bridge  the  gap  between  the  “art  and  science”  of 
effective  instruction  while  conducting  a  VBS2  mission  AAR.  The  results  of  the  exploratory 
investigation  suggest  that  employing  VBS2  AAR  capabilities  can  materially  enhance  Soldiers’ 
learning.  Cadre  and  facilitators  should  be  made  aware  of  the  added  value  of  employing  VBS2 
AAR  features  during  AARs  of  VBS2  training  exercises.  The  VBS2  AAR  Guide  is  currently  being 
transitioned  to  the  cadre  of  three  courses,  has  been  posted  on  the  MilGaming  community  of 
practice  website  for  downloading,  and  has  been  incorporated  into  the  Soldier ’s  Toolbox  for 
Developing  TTP.  Finally,  a  companion  research  product  has  been  published  to  facilitate  the 
transition  efforts. 
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GUIDELINES  AND  TOOLS  FOR  VBS2  MISSION  AFTER  ACTION  REVIEWS: 

DEVELOPMENT  AND  EVALUATION 

Introduction 

As  part  of  the  U.S.  Army’s  efforts  to  leverage  the  power  of  computer-based  combat 
simulations,  Army  trainers  are  embracing  desktop  simulations  such  as  the  Virtual  Battlespace-2 
(VBS2)  game.  This  virtual,  multi-player,  interactive  system  provides  a  realistic  tactical 
environment  for  training  collective  tasks.  Soldiers  and  leaders  execute  missions  that  are  driven 
by  operational  scenarios  and  then  reflect  on  their  performance  in  technology-supported  after 
action  reviews  (AAR).  In  the  VBS2  training  environment,  trainers  can  accelerate  the  AAR 
learning  process  by  harnessing  special  feedback  capabilities  of  the  software.  Given  the 
expanding  use  of  VBS2  throughout  the  Army,  the  need  has  emerged  for  a  tool  to  help  AAR 
facilitators  put  the  game’s  new  simulation-based  AAR  capabilities  into  practice.  The  U.S.  Army 
Research  Institute  (ARI)  responded  to  the  need  by  developing  tailored  AAR  guidelines  and  tools 
for  trainers  of  VBS2  missions. 

In  this  ARI  research,  investigators  built  on  previous  research  that  created  methods  and 
measures  for  developing  and  communicating  tactics,  techniques,  and  procedures  (TTP).  The 
earlier  research  led  to  the  construction  of  a  Soldier ’s  Toolbox  for  Developing  TTP  that 
incorporated  lessons  and  insights  gained  from  the  investigation  and  evaluation  efforts.  The 
Toolbox  gave  units  a  Soldier- friendly  package  to  help  them  capture  their  hard- won  combat 
lessons  and  share  them  with  newly  arrived  Soldiers.  The  present  research  expanded  the 
methodology  to  include  methods  for  communicating  TTP  in  the  specialized  VBS2  training 
environment. 

This  report  describes  the  methods  used  to  develop  and  evaluate  the  VBS2  AAR  Guide ,  a 
tool  that  aims  to  optimize  the  benefits  of  employing  the  VBS2  system’s  AAR  capabilities.  It 
explains  the  efforts  to  investigate  how  the  Guide  impacts  learning  processes  and  outcomes,  and 
presents  the  results  of  the  evaluation  and  investigation  work.  The  report  includes  a  complete 
copy  of  the  Guide  as  a  handy  reference.  A  companion  publication  (Green,  Leibrecht,  &  Fite,  in 
preparation)  describes  and  characterizes  the  VBS2  AAR  Guide,  and  explains  how  to  use  the 
Guide  in  optimal  fashion. 


Training  Support  Requirements 

The  requirements  underpinning  the  present  research  stem  from  training  and  development 
challenges,  feedback  dynamics  of  simulation-situated  AARs,  and  today’s  imperative  to  train 
adaptive  Soldiers. 

Training  and  Development 

As  today’s  operational  environment  (OE)  continues  to  transform,  Soldiers  who  can  adapt 
to  uncertain  and  unforeseen  challenges  have  a  distinct  advantage  (Haskins,  2009).  Soldiers  face 
an  unpredictable  and  adaptive  enemy,  which  brings  unforeseen  problems  that  require  capable 
problem  solvers.  Historically,  Army  leaders  geared  unit  training  around  relatively  stable  tasks 
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(Ferguson,  2008).  Soldiers  were  trained  to  meet  minimum  standards  of  performance  on  tasks 
which  they  could  efficiently  execute  in  a  predictable  environment.  This  strategy  is  no  longer 
optimal  because  current  missions  are  increasingly  complex  due  to  unpredictable  factors.  As 
Field  Manual  3-0  (U.S.  Department  of  the  Army,  2008)  explains,  Soldiers  now  operate  in  a  full 
spectrum  environment  that  requires  adaptive  thinking  and  careful  use  of  judgment. 

As  new  operational  requirements  and  technologies  are  integrated  into  the  U.S.  Army, 
Soldiers  and  units  are  often  on  their  own  to  determine  how  a  new  requirement  or  technology 
impacts  their  existing  TTP  and  tactical  standing  operating  procedures  (TSOP).  When  the  unit 
attempts  to  adapt  to  the  new  requirement  or  technology,  the  lack  of  workable  guidance  and  tools 
for  developing  new  TTP/TSOP  or  revising  existing  TTP/TSOP  reduces  the  unit’s  ability  to  keep 
pace  with  critical  changes.  The  pace  of  change  in  operational  requirements  and  available 
technology  is  accelerated  by  the  uncertainties  of  irregular  warfare  as  well  as  rapid  fielding  and 
modernization  initiatives.  For  these  reasons,  units  require  assistance  in  exploring,  developing, 
revising,  communicating,  and  assessing  TTP/TSOP  to  better  respond  to  the  dynamic  and 
unpredictable  nature  of  irregular  warfare. 

The  rapid  pace  of  change  across  the  force  has  affected  the  manner  in  which  TTP  are 
developed.  Instead  of  relying  on  centrally  developed  TTP,  tactical  units  typically  develop  TTP 
as  needed,  “on  the  fly”  with  little  formal  guidance  or  structure.  According  to  Army  convention 
(U.S.  Department  of  the  Army,  2005),  TTP  play  a  central  role  in  unit  operations,  including 
mission  planning  and  rehearsal.  However,  no  guidance  is  provided  on  how  to  develop  TTP.  A 
procedurally  sound,  cost  effective  and  Soldier-friendly  method  for  developing  TTP  is  required  to 
keep  pace  with  rapid  changes  in  warfighting  capabilities.  A  more  formal  way  of  creating  and 
documenting  TTP  may  increase  the  effectiveness  of  the  TTP  outcomes  and  facilitate  the 
documentation  of  “tribal  knowledge.”  Tribal  knowledge  consists  of  undocumented  procedures 
and  information  possessed  within  a  group,  but  not  readily  understood  to  individuals  outside  of 
the  group. 

The  AAR  as  Feedback  Forum 

Soldiers  require  feedback  about  what  happened  during  an  exercise.  Because  of  the  “fog 
of  war,”  phenomenon,  when  an  exercise  ends,  participants  sometimes  have  a  limited  perspective 
regarding  what  happened,  based  on  the  information  available  to  them  and  what  they  saw,  heard, 
and  smelled.  This  limited  perspective  is  referred  to  as  perceived  truth.  Ground  truth  is  the  term 
used  for  the  actual  events  that  occurred.  Less  trained  units  are  expected  to  demonstrate  a  greater 
disparity  between  perceived  and  ground  truth,  simply  because  much  of  the  intrinsic  information 
that  was  available  was  not  perceived  or  used.  Events  may  be  happening  quickly  and  are  open  to 
differing  interpretations.  Perceptions  and  memories  of  the  occurrence,  sequence,  and  timing  of 
events  can  be  greatly  distorted— leading  to  inference  of  causal  relationships  which  do  not  reflect 
the  actual  facts  (Goldberg  &  Meliza,  1993). 

Extrinsic  feedback  is  provided  by  outside  sources — usually  observer/controllers  or 
trainers — after  an  exercise  ends.  In  the  Army,  the  established  forum  for  orchestrating  extrinsic 
feedback  is  the  AAR.  It  is  designed  to  help  Soldiers  understand  the  ground  truth  situation 
relative  to  their  perceived  truth  and  to  investigate  what  caused  the  events  to  occur  as  they  did. 
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Extrinsic  feedback  exposes  exercise  participants  to  information  that  is  not  ordinarily  available  to 
them.  Soldier-participants  are  encouraged  to  identify  their  problems  and  develop  approaches  to 
correct  them.  Extrinsic  feedback  can  provide  insights  into  how  to  improve  or  sustain 
performance  in  the  future. 

Extrinsic  feedback  regarding  unit  performance  focuses  on  conceptual  knowledge  rather 
than  procedural  knowledge.  Such  feedback  is  generally  more  explanatory  than  directive  in 
nature.  Research  has  shown  that  explanatory  feedback  is  superior  to  directive  feedback  when  it 
comes  to  building  conceptual  knowledge  (Moreno,  2006).  As  described  in  Army  doctrine 
(Department  of  the  Army,  1993),  the  AAR  is  an  active  process  that  emphasizes  interactive 
discussion.  A  facilitator  or  trainer — the  AAR  leader — guides  the  discussion  to  focus  on  what 
happened,  why  it  happened,  and  how  to  improve  or  sustain  performance. 

Training  for  Adaptive  Performance 

Vandergriff  (2006)  defines  adaptability  as  “the  process  by  which  individuals  and  groups 
decide  rapidly,  almost  instinctively,  to  changes  in  their  situation”  (p.  43).  According  to 
Vandergriff,  a  Soldier’s  experience  in  problem-solving  exercises  helps  him  become  an  adaptive 
leader.  Such  experience  is  important  in  training  so  that  Soldiers  know  “what  right  looks  like.” 
When  faced  with  a  tactical  challenge,  what  is  the  best  course  of  action  to  take?  How  can 
Soldiers  accomplish  a  task  quickly  and  efficiently,  while  minimizing  costs  to  the  unit?  These 
questions  arise  routinely  and  take  on  added  significance  during  combat  operations,  where  costs 
include  life-and-death  risks  for  friendly  troops  and  civilians.  To  work  effectively  in  the  OE, 
Soldiers  need  skills  and  attributes  that  enable  them  to  solve  unfamiliar  problems  in  the  midst  of 
complex  conditions.  The  Army  must  explore  new  methods  of  training  and  education  to  produce 
warfighters  who  can  think  and  act  more  effectively  in  complex  environments. 

Background 


Previous  Relevant  Research 

Over  the  past  few  years,  ARI  has  addressed  the  need  for  innovative  methods  to  create  and 
share  TTP  by  conducting  a  series  of  focused  research  investigations.  In  the  initial  project 
(Topolski,  Leibrecht,  Kiser,  Kirkley,  &  Crabb,  2009),  ARI  investigators  developed  a  novel 
method  to  structure  the  TTP  development  process  for  emerging  systems  or  technologies.  Based 
on  Shadrick,  Lussier,  and  Hinkle’s  (2005)  flexible  process  of  cognitive  task  analysis,  the  TTP- 
focused  method  is  a  vignette-driven,  iterative  approach  that  harnesses  knowledge  elicitation 
techniques  with  groups  of  military  subject  matter  experts  (SME).  The  approach  was  found  to  be 
effective  in  developing  TTP,  but  it  does  not  lend  itself  well  to  independent  use  by  units  (Topolski 
et  al.,  2009).  To  extend  the  utility  of  the  new  method,  follow-up  work  produced  a  Soldiers  ’ 
Toolbox  for  Developing  TTP  that  units  can  use  in  table-top,  simulation,  and  live  training  modes 
to  develop  and  revise  TTP  (Topolski,  Leibrecht,  Porter,  Green,  Haverty,  &  Crabb,  2010).  The 
TTP  Toolbox  was  successfully  transitioned  to  the  Future  Force  Integration  Directorate  at  Fort 
Bliss  in  support  of  their  efforts  to  field  new  technologies  to  the  Army. 
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The  findings  of  that  early  research  led  to  the  realization  that  communicating  TTP  and 
ensuring  their  understanding  are  as  important  as  creating  and  maturing  the  TTP.  Accordingly, 
the  next  project  in  the  series  investigated  the  best  methods  for  communicating  TTP  and  for 
measuring  how  well  Soldiers  understand  them  (Anderson,  Topolski,  Leibrecht,  Green,  Crabb,  & 
Lickteig,  2010).  Three  modes  of  communication — written,  graphic  plus  written,  and  video  plus 
graphic  plus  written — were  developed  and  evaluated.  In  addition,  three  techniques  for  assessing 
Soldier  understanding  were  constructed — written  back  brief,  multiple-choice  questioning,  and 
problem  solving  based  on  video  modeling.  Those  findings  led  to  questions  about  implementing 
methods  for  communicating  TTP  in  conjunction  with  established  mechanisms  for  promoting 
Soldier  learning,  such  as  the  AAR.  This  set  the  stage  for  the  research  presented  here,  which 
advanced  the  line  of  investigation  into  the  realm  of  simulation-based  training. 

Demands  on  Trainers  in  Units  and  Institutions 

To  be  effective,  AAR  discussions  need  to  be  guided  by  a  leader  or  facilitator.  The 
warfighter  leading  an  AAR  needs  one  or  more  start  points  for  the  discussion  and  at  least  a 
general  idea  of  its  direction.  Then,  by  asking  open-ended  and  leading  questions,  the  AAR 
facilitator  gets  the  Soldier-trainees  to  relate  the  significant  events  that  happened  during  the 
exercise.  The  job  of  the  facilitator  becomes  easier  to  the  extent  that  he/she  is  already  aware  of 
the  types  of  problems  the  unit  has  been  experiencing.  If  all  an  AAR  leader  knows  about  a 
mission  is  that  the  unit  sustained  heavy  casualties,  questioning  will  take  a  long  time  to  identify 
the  root  causes  of  the  problem.  The  AAR  does  not  require  an  exhaustive  review  of  all  aspects  of 
a  unit’s  performance.  Instead,  trainers  should  focus  on  aspects  of  performance  closely  linked  to 
key  exercise  events  and  outcomes.  Thus,  AAR  facilitators  must  be  fully  knowledgeable,  well 
prepared,  and  equipped  with  tools  for  promoting  feedback  and  learning  among  the  participating 
Soldiers. 

Growth  of  Training  Simulations 

The  Army  faces  significant  training  challenges  now  and  in  the  years  ahead.  These 
challenges  include  the  persistently  high  operational  tempo,  deployment  pressures  on  training 
requirements  and  schedules,  and  the  Army  Force  Generation  process.  Transformation  of  the 
Army  structure,  the  complexity  of  potential  domestic  and  global  missions,  greater  diversity  of 
operational  environments,  and  competing  requirements  for  training  resources  are  all  key  factors 
that  pressure  current  and  future  training  realities.  Consequently,  trainers  are  relying  more  and 
more  on  low-cost,  simulation-based  solutions  to  fill  critical  training  gaps  and  increase  the 
effectiveness  of  training  for  our  Soldiers  in  a  time  of  persistent  war. 

The  U.S.  military  has  a  long  history  of  using  simulations  to  train  Soldiers.  For  example, 
Simulation  Networking  (SIMNET)  is  an  instructional  environment  designed  to  aid  in  training 
mounted  units.  The  SIMNET  is  a  virtual,  real-time,  multi-station  interactive  network.  The 
second  generation  of  SIMNET  emerged  as  the  Close  Combat  Tactical  Trainer  (CCTT).  The 
CCTT  system  incorporated  several  improvements  over  the  SIMNET  suite  including  the  ability  to 
vary  light  and  weather  conditions.  A  limitation  of  both  SIMNET  and  CCTT  is  their  bulk, 
restricting  their  portability.  Leveraging  personal  computer  (PC)  capabilities,  DARWARS 
Ambush!  offered  a  “lightweight,”  commercial  off-the-shelf  training  simulation  that  has  been 
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used  widely  within  the  Army.  Recently,  VBS2  was  fielded  as  a  second  generation  version  of 
DARWARS  Ambush!  and  is  currently  being  employed  throughout  the  Army.  While  these 
systems  vary  along  multiple  dimensions  (e.g.,  fidelity,  sophistication,  terrain  representation), 
they  all  share  the  ability  to  train  Soldiers  and  support  TTP  development. 

Role  of  Simulations 

Currently,  Soldiers  train  with  the  help  of  simulations  for  a  wide  range  of  skills  including 
working  as  a  team,  commanding  and  controlling  operations,  making  decisions,  strategizing,  and 
operating  costly  equipment  (e.g.,  Prensky,  2007).  Overall,  training  in  simulation  environments 
should  provide  individuals  and  units  with  feedback  about  how  their  actions  contributed  to 
mission  success  or  failure,  casualties  received,  and  casualties  inflicted  on  the  enemy— the  bottom 
lines  of  collective  performance.  According  to  Prensky  (2007),  simulations  may  also  be  used  to 
expand  the  range  of  feasible  conditions  in  which  units  can  train,  reduce  the  requirements  for 
training  ranges,  avoid  the  maintenance  burden  of  live  training,  and  bring  greater  flexibility  to 
unit  training  schedules.  Simulations  also  can  provide  high-fidelity  realism  and  practice  in 
competitive  situations. 

Costs/Benefits  of  Simulations 

Prensky  (2001)  suggests  that  the  benefits  of  simulations  include  lower  cost,  increased 
motivation  of  Soldiers,  and  greater  operational  relevance.  Simulators  are  cheaper  than  using 
operational  equipment,  an  advantage  that  applies  especially  to  aircraft  (Hays,  Jacobs,  Prince,  & 
Salas,  1992).  However,  the  initial  cost  of  creating  and  validating  a  game-based  simulation  can 
be  substantial.  For  example,  the  U.S.  Army  invested  considerable  resources  in  establishing  the 
CCTT  as  a  robust  virtual  trainer  for  team  and  collective  tasks  (Mastaglio  et  al.,  2004;  U.S.  Army 
Program  Executive  Office  for  Simulation,  Training,  and  Instrumentation,  2010).  Once  the 
training  simulation  has  been  established,  the  cost  of  training  in  a  virtual  battlefield  is  less  than  it 
would  be  when  using  actual  equipment  on  training  ranges.  In  comparison  to  both  live  training 
and  virtual  simulation,  game-based  training  (GBT)  platforms  are  the  least  costly  option.  As  an 
example,  the  multi-player,  internet-based  game  Spearhead  II  trains  crews  on  artillery  fire  control, 
and  it  simulates  mobility  and  combat  interactions  for  25  cents  an  hour  per  player  (Erwin,  2000). 
Dome-based  simulators  with  a  motion-base  and  wrap-around  imagery  cost  $5,000  to  $10,000  to 
operate  for  each  hour  of  useful  training. 

Several  studies  support  the  notion  that  simulations  increase  learners’  motivation  (e.g., 
Moreno,  2006;  Prensky,  2007).  Massively  multiple  online  role-playing  games  (MMORPG)  are 
synthetic  arenas  where  players  interact,  collaborate,  and  strategize  with  others.  The  nature  of 
MMORPGs  challenges  players  to  think  critically  and  plan  ahead,  while  promoting  intrinsic 
motivation  through  choice,  control,  collaboration,  challenge,  and  achievement  (Dickey,  2007). 
Using  games  as  instructional  tools  can  increase  motivation  to  learn  by  more  fully  engaging  the 
training  audience  (Vogel,  Greenwood-Ericksen,  Cannon-Bowers,  &  Bowers,  2006).  These 
findings  are  reinforced  by  America’s  Army,  a  game  developed  as  a  recruiting  tool  to  inform  the 
eligible  population  about  military  service  as  a  Soldier  (http://www.americasarmy.com).  The 
game  is  a  PC-based  first-person-perspective  game  where  players  go  through  virtual  basic  training 
and  then  complete  online  military  missions  as  part  of  a  team.  As  evidence  of  the  game’s 
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motivational  allure,  figures  posted  on  the  game’s  website  indicate  there  are  over  10  million 
registered  players,  with  5.4  million  having  completed  basic  training. 

Regarding  operational  relevance,  preparing  small  teams  of  dismounted  infantry  for  urban 
operations  is  one  of  the  greatest  military  training  challenges  today  (Lampton,  Clark,  &  Knerr, 
2003).  As  Lampton  and  his  colleagues  note,  GBT  “supports  small  unit  (fire  team,  squad,  and 
platoon)  training,  mission  rehearsal,  and  explorations  and  evaluation  of  potential  changes  in 
doctrine,  organizations,  equipment,  and  Soldier  characteristics”  (Lampton  et  al.,  2006,  p.l).  The 
technology  also  imposes  real-time  pacing,  vehicle  and  weapon  operating  demands,  movement 
and  maneuver  activities,  crew  interaction,  command  and  control  demands,  terrain  complexities, 
and  a  battlefield  environment  rich  in  visual  and  audio  cues.  The  GBT  venue  compels  Soldiers  to 
apply  their  tactical  knowledge  under  dynamic  conditions  that  emulate  the  OE. 

After  Action  Reviews 


Description  of  AARs 

According  to  Morrison  and  Meliza’s  (1999)  comprehensive  analysis,  the  genesis  of  the 
modem  AAR  may  be  traced  to  historian  S.  L.  A.  Marshall,  who  documented  events  during 
World  War  II  and  the  Korean  and  Vietnam  Wars.  Marshall  spoke  with  Soldiers  in  theater  and 
immediately  after  combat  actions.  His  efforts  were  arguably  the  first  AARs.  The  lack  of  modem 
technology  restricted  first-hand  observations  and  documentation  of  battles,  as  reporters  were 
limited  by  space  and  time. 

The  modem  day  AAR  process  evolved  in  parallel  with  tactical  engagement  simulations. 
As  Morrison  and  Meliza  (1999)  note,  AARs  involve  after-the-fact  discussion  of  an  exercise 
where  participants  can  consider  the  events  from  multiple  perspectives  and  come  to  an  individual 
understanding  of  three  key  issues: 

1 .  What  happened  during  the  collective  training  exercise?  The  AAR  participants  attempt  to 
recount  the  facts  (i.e.,  the  important  actions  and  outcomes)  of  the  simulated  battle. 

2.  Why  did  it  happen?  Given  the  facts  of  the  exercise,  the  participants  attempt  to  explain 
the  causes  of  critical  actions  and  outcomes. 

3.  How  can  the  unit  improve  its  performance?  Once  the  first  two  questions  are  answered, 
the  participants  determine  appropriate  actions  to  remedy  notable  performance  problems. 
Example  actions  include  changing  the  unit’s  standing  operating  procedures  (SOP)  or 
conducting  further  training  with  basic  drills. 

Purpose  of  AARs 

Bringing  together  contrasting  accounts  is  often  the  key  element  in  an  AAR.  In  a  team 
exercise,  individuals  view  success  and  failure  through  the  lens  of  their  individual  perceptions  and 
decisions.  However,  given  the  distributed  nature  of  these  exercises,  an  individual’s  perspective 
is  frequently  biased  or  misleading.  For  example,  what  seems  like  a  bad  decision  from  the 
perspective  of  a  subordinate  might  make  more  sense  if  he  were  placed  in  his  commander’s  shoes. 
By  collectively  discussing  events  after  the  mission  ends,  decision  makers  can  come  to  a  truer 
understanding  of  who  did  what,  why,  and  how  they  can  do  better  the  next  time.  Based  on  greater 
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understanding  and  objective  insights,  leaders  can  improve  their  unit  performance  by  developing 
or  refining  their  TTP  and  ensuring  that  their  Soldiers  thoroughly  understand  the  new  TTP. 

Benefits  of  AARs 

Well  structured  AARs  provide  units  with  an  improved  perspective  on  what  actually 
happened  during  an  exercise — a  perspective  that  more  accurately  reflects  ground  truth.  An 
important  goal  of  the  unit  is  to  identify  corrective  actions  to  improve  every  Soldier’s  perspective 
as  training  and  operations  proceed,  thereby  generating  better  intrinsic  feedback  to  cue  and  guide 
unit  behavior. 

Morrison  and  Meliza  (1999)  provided  part  of  the  reason  for  the  AAR’s  success  by  tracing 
its  basis  in  behavioral  science  principles.  Those  principles  include  feedback,  performance 
measurement,  memory,  group  interaction,  information  sharing,  and  instruction.  Objective 
standards  and  computer-based  data  collection  ensure  measureable  performance.  Feedback 
includes  execution  data  and  options  for  changing  or  improving  future  activities.  This  occurs  as 
the  AAR  facilitator  coaches  the  unit  through  self-discovery.  Immediate  memory  plays  a  key  role 
during  an  AAR,  while  take  home  packages  provide  reference  materials  for  later  study.  Group 
problem  solving  techniques  help  units  to  understand  what  happened  and  how  to  make  changes. 
Interpersonal  interactions  come  from  cohesive  groups  sharing  individual  views  in  a  non¬ 
threatening  environment  with  an  AAR  facilitator.  Guided  by  the  facilitator,  the  unit  identifies  its 
own  fixes  as  its  Soldiers  engage  in  active  learning  (see  Morrison  &  Meliza,  1999,  p.  24-25,  for  a 
comprehensive  discussion).  In  theory,  then,  the  AAR  is  a  continuous  learning  process  reflecting 
the  desire  to  improve/sustain  performance  or  the  need  to  change  behavior  in  order  to  achieve 
more  favorable  outcomes. 


Research  Objectives 

The  current  research  aimed  to  develop  innovative  methods  and  materials  to  help  trainers 
enhance  the  effectiveness  of  VBS2  AARs.  The  products  of  the  research  were  intended  for  use 
by  AAR  facilitators  in  institutional  and  unit  settings.  An  underlying  goal  was  to  extend  the  reach 
of  the  previously  constructed  Soldier ’s  Toolbox  for  Developing  TTP  into  the  simulation-based 
training  environment.  The  following  research  objectives  guided  our  efforts: 

1.  Survey  best  practices  that  are  applicable  to  simulation-situated  AARs. 

2.  Translate  best  practices  into  guidelines  and  tools  to  support  AARs  of  VBS2  missions. 

3.  Package  the  guidelines  and  tools  in  a  facilitator- friendly  guide. 

4.  Formatively  evaluate  the  guide  in  institutional  settings. 

5.  Investigate  the  guide’s  impact  on  AAR  processes  and  training  outcomes. 

The  VBS2  was  chosen  as  the  gaming  platform  of  interest  for  several  reasons.  First,  it  is 
widely  used  by  Army  organizations  including  the  Training  and  Doctrine  Command’s  Centers  of 
Excellence.  Second,  VBS2  is  capable  of  supporting  mounted  and  dismounted  operations.  Third, 
VBS2  provides  a  robust  family  of  AAR  capabilities  including  recording  and  playback, 
bookmarking,  flexible  camera  views,  and  other  features.  Finally,  VBS2  enables  trainers  to 
readily  vary  OE  factors  such  as  mission,  friendly  troops,  enemy,  and  terrain. 
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Method 


The  purpose  of  this  research  was  to  develop  a  Guide  to  assist  facilitators  with  applying 
VBS2  AAR  capabilities  during  AARs  as  part  of  simulation-based  training.  The  development 
process  called  for  a  formative  evaluation  of  the  Guide.  In  addition,  the  research  measured  the 
impact  of  using  VBS2  AAR  capabilities  on  AARs  and  instructor  and  student  perceptions  of 
simulation-based  training.  Construction  of  the  Guide  began  with  a  front-end  analysis  of  the  best 
practices  from  industry,  academia,  the  military,  and  VBS2  circles.  The  results  of  the  analysis 
were  used  to  develop  educational  materials,  procedural  guidelines,  and  a  job  aid.  The  resulting 
Guide  underwent  operational  implementation  and  multiple-stage,  multi-source  evaluation  with 
representatives  of  the  target  audience. 


Front-End  Analysis 

An  extensive  review  of  the  U.S.  Army’s  AAR  doctrine,  research  and  technical  articles 
from  academia  and  industry,  and  VBS2  training  manuals  was  conducted.  Army  doctrine  was 
examined  to  ensure  the  Guide  would  be  consistent  with  the  established  structure  and  practices  of 
an  AAR.  Research  articles  were  reviewed  to  identify  best  practices  and  avoid  common  pitfalls 
and  shortcomings  in  constructing  the  Guide.  Training  manuals  for  the  VBS2  system  were  used 
to  specify  VBS2  capabilities,  terminology  and  system  requirements. 

Interviews  were  conducted  with  representatives  of  the  target  audience.  Limited  to 
selected  training  institutions,  the  target  audience  representatives  included  course  managers, 
instructors  and  SMEs.  The  interviews  focused  on  determining  user  needs  and  preferences  for  the 
Guide.  Observational  data  was  recorded  during  course-related  AARs  to  gather  additional 
information  about  facilitators’  current  practices.  Finally,  the  research  team  also  examined  the 
available  documents  and  presentation  materials  from  lectures  and  workshops  pertaining  to  VBS2 
AARs.  Selection  of  the  lectures  and  presentations  was  based  on  the  military  background, 
simulation  experience,  and  favorable  third-party  comments  regarding  the  presenter(s). 

Initial  Development  of  the  Guide 

The  results  of  the  front-end  analysis — doctrine,  best  practices,  principles,  lessons  learned, 
etc. — was  used  to  prepare  guidelines  for  conducting  AARs  with  the  aid  of  a  VBS2  workstation. 
Several  sources  from  the  literature  review  provided  direct  support  in  this  step.  For  example, 
Training  Circular  25-20,  A  Leader's  Guide  to  After-Action  Reviews  (Department  of  the  Army, 
1993),  outlined  techniques  and  procedures  for  conducting  a  sound  AAR.  Several  other 
documents  (e.g.,  Allen  &  Smith,  1994;  Meliza,  1996,  1998;  Salter  &  Klein,  2007)  contributed 
useful  accounts  of  the  goals,  structure  and  processes  of  an  AAR.  The  VBS2  Administrator 
Manual  (Bohemia  Interactive  Australia,  2010)  was  used  to  outline  the  VBS2  capabilities  and 
limitations,  as  well  as  the  workstation  requirements.  Assembled  in  outline  form,  the  guidelines 
were  reviewed  by  our  team  and  revised. 

Based  on  the  revised  guidelines,  a  concise  facilitator’s  Guide  containing  orientation  and 
educational  materials,  procedural  guidelines  organized  by  AAR  stage,  and  a  job  aid  that  listed 
the  AAR  capabilities  of  the  VBS2  system  was  constructed.  Also  included  was  a  list  of  sources 
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of  additional  information.  As  part  of  the  initial  development,  multiple  reviewers  vetted  the  draft 
versions.  Following  each  review,  revisions  of  the  Guide  were  accomplished  as  needed. 

Formative  Evaluation 


Stages  and  Events 

The  formative  evaluation  was  structured  around  two  broadly  defined  stages.  The  first 
stage  consisted  of  iterative  vetting  of  the  Guide.  The  vetting  was  performed  by  SMEs  and 
academic  experts  to  ensure  the  Guide  adhered  to  AAR  doctrine,  contained  correct  VBS2 
terminology,  accurately  conveyed  VBS2  capabilities  and  limitations,  sufficiently  outlined  the 
necessary  skills  and  required  equipment,  and  comprehensively  outlined  the  desired  steps  for 
conducting  an  AAR  with  VBS2  tools.  The  Guide  was  then  reviewed  by  a  behavioral  scientist 
using  the  same  criteria.  Following  each  vetting  cycle,  the  Guide  was  revised  based  upon  the 
feedback. 

In  stage  two,  the  Guide  was  field  tested  by  instructors  in  three  separate  courses  at  two 
U.S.  Army  schoolhouses.  Instructors  were  asked  to  evaluate  the  acceptability  and  usability  of 
the  Guide.  In  addition,  feedback  was  solicited  on  which  sections  could  be  clarified,  expanded, 
reduced  or  deleted.  The  testing  procedure  occurred  in  multiple  cycles,  with  each  group  of 
instructors  given  several  opportunities  to  provide  feedback  on  the  Guide. 

Participants 

The  participants  for  the  formative  evaluation  were  current  course  instructors  including 
both  active  military  and  contractor  SMEs.  Five  instructors  provided  written  and/or  verbal 
feedback  regarding  the  Guide. 

Materials 

Feedback  data  was  collected  from  participants  using  the  Instructor  Feedback  Form  and 
the  Instructor  Profile  and  Survey  (see  Appendix  B).  The  forms  contained  a  mix  of  open-ended, 
rating  scale,  and  check-all-that-apply  questions.  The  questions  focused  on  three  main  topics:  the 
acceptability  of  the  Guide,  the  usefulness  of  the  Guide,  and  suggested  changes  to  the  Guide. 

Data  Collection  Procedures 

Participants  were  given  a  hardcopy  of  the  Guide  to  review,  along  with  the  Instructor 
Feedback  Form  to  complete.  They  were  asked  to  thoroughly  review  the  Guide  and  provide  any 
comments  or  suggestions  they  felt  appropriate.  While  the  Guide  was  distributed  to 
approximately  three  dozen  participants,  feedback  forms  were  received  from  only  three.  This 
reflected  the  time  pressures  and  competing  priorities  of  the  institutional  environment.  The 
Instructor  Profile  and  Survey  was  administered  to  course  instructors  immediately  following  an 
AAR  with  VBS2.  The  survey  took  approximately  5  min  to  complete. 
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Exploratory  Investigation 


An  exploratory  investigation  was  performed  to  assess  the  effects  of  the  Guide  and  VBS2 
AAR  capabilities  on  learning  outcomes  and  the  AAR  process.  The  sampling  domain  included 
instructors  and  students  in  three  Army  courses.  The  investigation  design  for  instructors  was  a 
self-selection,  small  sample  model,  emphasizing  descriptive  data.  The  investigation  design  for 
students  was  a  between-group  model  with  a  non-random  sample.  The  independent  variable 
focused  on  the  use  or  non-use  of  VBS2  capabilities  during  the  AAR  session:  the  experimental 
condition  was  defined  as  AAR  with  VBS2,  while  the  control  condition  was  defined  as  AAR  with 
no  VBS2.  The  Guide  was  distributed  to  all  of  the  instructors,  both  experimental  and  control,  at 
least  one  week  prior  to  conducting  their  AARs.  Multiple  sources  of  data  included  hand-recorded 
observations,  written  and  verbal  feedback,  and  psychometrically  scaled  ratings. 

Participants 

Instructors  and  students  were  drawn  from  courses  taught  by  (a)  the  Maneuver  Center  of 
Excellence  at  Fort  Knox,  Kentucky,  and  (b)  the  Maneuver  Support  Center  of  Excellence  at  Fort 
Leonard  Wood,  Missouri.  Instructor  and  student  participants  came  from  two  courses  at  Fort 
Knox:  the  Advanced  Leaders  Course  (ALC)  for  Cavalry  Scouts  and  the  Army  Reconnaissance 
Course  (ARC).  At  Fort  Leonard  Wood  the  participants  consisted  of  instructors  from  the  Basic 
Officer  Leaders  Course  (BOLC)  for  Engineer  Corps  officers.  This  population  was  chosen 
because  the  courses  already  integrated  VBS2-based  training  within  their  program  of  instruction, 
and  the  training  missions  involved  mounted  and  dismounted  operations.  As  is  typical  for  such 
courses,  the  instruction  focused  on  teaching  TTP  and  TSOP  at  the  small  unit  level. 

Numerous  factors  limited  the  instructors’  participation.  Meetings  and  other  regularly 
scheduled  course  activities  prevented  some  instructors  from  completing  the  data  collection 
instruments  following  the  AARs.  In  addition,  the  ALC  program  was  relocating  to  another 
installation,  producing  extra  time  pressures  on  instructors.  As  a  result,  the  quantity  of  data 
collected  was  uneven  across  measuring  instruments.  Some  instructors  completed  only  the 
instructor  feedback  survey,  while  others  participated  in  the  hotwash  only  (i.e.,  did  not  complete 
an  instructor  survey). 

The  effective  sample  size  was  further  reduced  by  the  level  of  participation  exhibited  by 
instructors  during  the  AAR.  While  two  instructors  were  always  physically  present,  one  led  the 
AAR  while  the  second  typically  performed  other  duties  (e.g.,  completing  paperwork,  planning 
the  next  exercise,  or  other  duties  not  directly  related  to  the  AAR).  The  research  team  concluded 
that  collecting  data  from  the  secondary  instructor  would  not  be  valid. 

Three  instructors  completed  the  Instructor  Profile  and  Survey,  all  from  the  experimental 
condition.  The  average  length  of  military  service  for  the  instructors  was  213  months  ( SD  = 
151.40).  The  average  length  of  time  as  an  instructor  for  the  current  course  was  14  months  ( SD  = 
3.46).  Prior  experience  with  VBS2  training  varied  from  “some  experience”  to  “much 
experience,”  while  prior  experience  with  other  simulations  such  as  DARWARS  Ambush!  was 
consistent  for  all  instructors  at  “some  experience.” 
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Data  was  collected  from  55  students  in  the  ARC  (n  =  17)  and  ALC  (n  =  38)  courses  at 
Fort  Knox.  Data  from  two  students  was  dropped  from  the  analysis  since  both  students  were  not 
U.S.  Soldiers  and  indicated  they  did  not  fully  understand  English.  This  resulted  in  a  final  sample 
of  53  students.  The  students  were  not  randomly  assigned  to  either  condition;  assignment  was 
based  upon  the  instructors’  willingness  to  conduct  the  AAR  utilizing  the  information  provided  in 
the  Guide  and  the  VBS2  AAR  capabilities.  Of  the  53  students,  31  served  in  the  experimental 
condition  (AAR  with  VBS2)  and  22  in  the  control  condition  (AAR  with  no  VBS2). 

Biographical  data  was  examined  for  any  differences  between  the  groups.  An  independent 
t-test  revealed  no  significant  difference  in  length  of  service  (months)  between  students  in  the 
experimental  condition  ( M=  92.52,  SD  =  55.8)  and  the  control  condition  (M=  79.55,  SD  = 
56.56).  As  seen  in  Figure  1,  the  experimental  condition  contained  a  higher  percentage  of 
sergeants,  while  the  control  condition  contained  an  even  mix  of  sergeants  and  lieutenants. 


SGT  SSG  1SG  2LT  1LT 

Military  Rank 


Figure  1.  Distribution  of  students’  military  rank,  by  research  condition. 

Biographical  data  was  also  examined  for  students’  prior  experience  with  VBS2  and  other 
simulation-based  training.  As  shown  in  Figures  2  and  3,  respectively,  the  majority  of  students 
reported  no  previous  experience  with  VBS2  or  other  simulations  prior  to  the  course.  Those  data 
suggest  that  subsequent  student  ratings  for  VBS2  and  simulation-based  training  could  largely  be 
attributed  to  their  experiences  in  the  current  courses  and  the  experimental  manipulation,  as 
opposed  to  past  experiences.  In  addition,  students  in  experimental  and  control  conditions 
appeared  to  have  very  simliar  levels  of  prior  experience  with  VBS2  and  other  simulations, 
indicating  that  both  groups  were  comparable  on  this  measure. 
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None  Little  Some  Much 

Experience  Experience  Experience 


Level  of  Experience 


Figure  2.  Students’  prior  experience  with  VBS2,  by  research  condition. 


None  Little  Some  Much 

Experience  Experience  Experience 

Level  of  Experience 

Figure  3.  Students’  prior  experience  with  computer  based  simulations,  by  research  condition. 
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Data  Collection  Materials 


To  assess  the  effects  of  incorporating  VBS2  AAR  capabilities  during  AARs,  multiple 
types  of  data  were  collected.  An  observation  guide  was  used  to  record  the  frequency  and  types 
of  interactions  and  behaviors  of  instructors  and  students.  Instructor  and  student  questionnaires 
were  developed  to  gather  biographical  data  and  measures  of  participants’  impressions  of  the 
effects  of  the  Guide  and  VBS2  AAR  capabilities  on  AARs  and  student  learning.  The 
questionnaires  contained  Likert-type  scales  and  open-ended  questions.  All  Likert  measures  were 
five  point  scales,  with  “1”  representing  “strongly  disagree,”  “3”  being  “neutral,”  and  “5”  being 
“strongly  agree.”  Higher  scores  translate  to  more  positive  ratings.  A  hotwash  protocol  was 
created  to  structure  interviews  with  instructors  in  order  to  obtain  additional  qualitative  data.  All 
of  the  data  collection  instruments  are  provided  in  Appendix  B. 

Procedures 

The  exploratory  investigation  tapped  multiple  sources  of  data  to  provide  converging 
evidence  of  the  effects  of  the  Guide  and  VBS2  AAR  capabilities  on  learning  outcomes  and  the 
AAR  process.  Instructors  received  the  Guide  as  much  as  two  weeks  prior  to  their  AAR,  but  they 
typically  studied  the  Guide  and  prepared  for  the  AAR  shortly  before  the  training  event.  During 
the  AAR,  one  of  the  research  team’s  trained  observers  used  3  or  5  min  intervals  to  record  the 
frequency  and  types  of  interactions  and  behaviors  for  instructors  and  students.  The  average 
duration  of  the  AARs  was  23.00  min  ( SD  =  7.21)  in  the  experimental  condition  and  33.33  min 
(iSZ)  =  .58)  in  the  control  condition.  The  difference  can  be  attributed  to  the  AAR  with  VBS2 
conducted  at  Fort  Leonard  Wood,  which  lasted  approximately  half  as  long  (15  min)  when 
compared  to  the  other  AARs. 

For  instructors,  a  single  subject  design  was  employed,  while  a  between-group  design  was 
employed  for  students.  Immediately  following  the  AAR,  the  instructors  and  students  were  given 
their  respective  questionnaires,  which  took  approximately  5  min  to  complete.  Afterward,  the 
students  were  dismissed  and  the  researchers  spent  approximately  20  min  conducting  the  hotwash 
with  instructors  using  the  protocol.  Instructors’  verbal  comments  were  recorded  and  transcribed 
by  the  interviewers. 


Results  and  Discussion 
Formative  Evaluation  Results 

In  the  formative  evaluation  of  the  Guide,  feedback  from  course  instructors  and  SMEs 
served  as  the  primary  data  source  and  provided  valuable  information  to  direct  refinement  efforts. 
The  feedback  gathered  during  the  evaluation  was  generally  positive.  Comments  and  suggestions 
for  revisions  were  relatively  minor  and  were  incorporated  into  the  final  version  of  the  Guide. 

The  formative  evaluation  results  will  be  organized  according  to  three  categories:  acceptability  of 
the  Guide,  utility  of  the  Guide,  and  suggested  improvements  to  the  Guide. 
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Acceptability  of  the  Guide 

Data  collected  from  Likert  scales  indicated  instructors  found  the  Guide  to  be  well  written 
and  appropriately  detailed.  As  seen  in  Table  1,  ratings  were  uniformly  positive  for  all  aspects  of 
the  Guide’s  quality.  This  data  was  consistent  with  the  instructors’  and  SMEs’  written  and  verbal 
comments  regarding  the  acceptability  of  the  Guide.  Instructors  stated  the  Guide  was  “very  clear 
and  understandable”  and  “easy  to  read.” 

Table  1 

Instructor  Ratings  of  Acceptability  of  VBS2  AAR  Guide 


How  much  do  you  agree  or  disagree  that  the  VBS2 
AAR  Guide : 

Instructor 

1 

Instructor 

2 

Instructor  ... 

^  Mean  (SB) 

Is  clearly  written? 

4 

5 

4 

4.33  (.58) 

Contains  the  right  level  of  detail  for  facilitators? 

4 

5 

4 

4.33  (.58) 

Avoids  unnecessary  or  excess  information? 

4 

5 

4 

4.33  (.58) 

Is  well  organized  and  easy  to  follow? 

4 

5 

4 

4.33  (.58) 

Uses  language  familiar  to  facilitators? 

4 

5 

4 

4.33  (.58) 

Note.  The  rating  scale  ranged  from  1  (strongly  disagree)  to  5  (strongly  agree).  Figures  in  parentheses  are 
standard  deviations. 


Utility  of  the  Guide 

Written  and  verbal  feedback  was  used  to  assess  the  perceived  usefulness  of  the  Guide. 
Instructors’  responses  indicated  that  they  believed  the  Guide  was  very  useful.  All  of  the  ratings 
were  positive  and  more  than  half  of  them  yielded  the  maximum  rating  of  “5.”  Instructor 
comments  echoed  the  positive  ratings  for  the  utility  of  the  guide  (e.g.,  “this  can  be  a  very  useful 
tool”).  But  the  comments  also  reflected  some  caution  regarding  how  the  VBS2  AAR  functions 
should  be  used  (e.g.,  “the  playback  should  not  become  the  focus  of  the  AAR”). 

Table  2 

Instructor  Ratings  of  Utility  of  VBS2  AAR  Guide 


How  much  do  you  agree  or  disagree  that  the  VBS2 
AAR  Guide: 

Instmctor 

1 

Instmctor 

2 

Instmctor 

3 

Mean  (SB) 

Is  a  useful  aid  for  incorporating  VBS2  tools  into  an 
AAR? 

4 

5 

4 

4.33  (.58) 

Is  a  valuable  addition  to  conduct  VBS2  exercises? 

4 

5 

5 

4.67  (.58) 

Can  help  facilitators  achieve  learning  outcomes? 

4 

5 

5 

4.67  (.58) 

Note.  The  rating  scale  ranged  from  1  (strongly  disagree)  to  5  (strongly  agree).  Figures  in  parentheses  are 
standard  deviations. 
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Suggested  Improvements  to  the  Guide 

Instructors’  written  and  verbal  feedback  did  not  suggest  the  need  for  major  modifications 
to  the  Guide.  As  a  general  rule,  instructors  indicated  the  Guide  was  “fine  as  is,  nothing  should 
be  changed”  and  “it  is  a  well  put  together  Guide,  I  don’t  know  how  you  would  improve  it.”  One 
instructor  recommended  the  use  of  a  laminated  pocket-sized  version  of  the  Guide  to  allow  for 
easier  use  during  the  AAR. 


Investigation  Results 


Characteristics  of  AARs 

Observational  data  was  recorded  during  the  AARs  to  quantify  the  frequency  and  types  of 
activities.  The  data  suggested  that  AARs  involving  VBS2  capabilities  significantly  influenced 
the  frequency  and  types  of  actions  performed  by  instructors  and  students.  As  shown  in  Figure  4, 
both  instructors  and  students  engaged  in  a  wider  array  of  behaviors  in  the  AAR  with  VBS2 
condition  than  the  AAR  with  no  VBS2  condition.  Students  were  also  more  active  participants  in 
the  AAR  with  VBS2  condition.  While  students’  behaviors  accounted  for  55%  of  recorded 
behaviors  in  the  AAR  withVBS2  condition,  compared  to  51.6%  in  the  ARR  with  no  VBS2 
condition,  the  difference  was  not  statistically  significant,  fl  (1,  N=  104)  =  .12,  p  =  .73.  The 
relatively  small  sample  size  and  infrequency  of  some  behaviors  made  a  more  refined  analysis 
impossible.  For  instance,  several  behaviors  were  not  observed,  such  as  “student  writes”  and 
“student  sketches.”  Observers’  notes  also  indicated  that  a  greater  number  of  students  actively 
participated  in  the  AAR  with  VBS2  condition.  Approximately  1-2  students  did  not  contribute 
comments  during  each  AAR  with  VBS2,  while  2-3  failed  to  contribute  during  each  AAR  with  no 
VBS2. 

Impact  of  the  Guide  and  VBS2  AAR  Capabilities  on  Training  Outcomes 

Observational  data  indicated  that  the  Guide  played  a  central  role  in  how  instructors 
employed  VBS2  AAR  capabilities  to  achieve  training  outcomes  during  AARs.  In  accordance 
with  the  Guide,  instructors  in  the  AAR  with  VBS2  condition  consistently  used  the  2-dimensional 
and  3-dimensional  map  views  to  review  the  commanders’  intent  and  mission  at  the  beginning  of 
the  AAR.  Bookmarking  was  used  as  an  editing  tool  to  compress  the  time  spent  reviewing  the 
critical  events.  Playback  views  were  varied  to  capitalize  on  the  facilitator’s  ability  to  review 
individual  and  group  actions.  Three  noteworthy  examples  will  be  discussed: 

1 .  One  facilitator  varied  camera  angles  between  map  view,  first-person  view,  and  free 
camera  view  to  illustrate  troop  movements  relative  to  the  enemy.  By  alternating  between 
these  views,  he  helped  Soldiers  to  realize  that  some  of  their  actions  diverged  from  the 
commander’s  intent. 

2.  Facilitators  used  the  first-person  view  during  dismounted  maneuvers  to  hold  students 
accountable  for  their  actions.  In  one  instance  of  fratricide,  the  student  claimed  from  his 
viewpoint  that  the  victim  looked  like  enemy.  The  facilitator  used  the  first-person  view  to 
demonstrate  that  from  the  student’s  vantage  point  the  victim  could  be  clearly  identified  as 
a  friendly  civilian.  In  another  instance  a  Soldier  did  not  take  the  exercise  seriously  and 
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ignored  the  mission.  The  facilitator  was  able  to  use  the  first-person  view  from  the 
enemy’s  perspective  to  illustrate  how  that  Soldier’s  actions  gave  away  the  unit’s  position 
and  endangered  his  unit. 

3.  A  facilitator  used  the  free  camera  and  first-person  views  to  illustrate  how  a  mounted  unit 
failed  to  successfully  recon  an  objective.  In  the  scenario,  the  Soldiers  drove  past  a 
motorcycle  parked  against  a  tree  in  the  woods  -  failing  to  see  it,  even  though  it  was  in 
their  line  of  sight.  The  motorcycle  provided  a  vital  cue  that  enemy  elements  were  in  the 
area.  Without  VBS2’s  AAR  capabilities,  it  would  more  difficult  for  the  facilitator  to 
review  with  Soldiers  this  critical  cue.  This  facilitator  also  used  the  first-person  view  to 
discuss  with  Soldiers  how  they  could  have  improved  their  line  of  sight  by  positioning 
their  tank  in  a  tactically  superior  position. 


Instructor  Instructor  Instructor  Instructor  Student  Student  Students 
Describes  Queries  Sketches  Fixes  Describes  Queries  Interact 

or  Writes  VBS2 
Problem 

Participant  Activities 

Figure  4.  Participants’  behaviors  during  the  AAR,  by  research  condition. 

The  instructor  and  student  feedback  surveys  contained  Likert  scale  items  and  open-ended 
questions  for  written  feedback  designed  to  assess  the  impact  of  the  Guide  and  VBS2  AAR  tools 
on  AARs.  The  items  were  derived  from  the  established  objectives  of  an  AAR  (Department  of 
the  Army,  1993).  The  data  from  instructors,  which  is  descriptive  and  represents  a  small  sample, 
is  presented  separately  from  the  student  data,  which  is  inferential  and  represents  a  larger  sample. 
As  appropriate,  between-group  comparisons  are  presented  in  the  following  sections. 
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Instructor  data 


Only  data  from  instructors  who  conducted  AARs  withVBS2  was  collected,  since 
instructors  in  the  AAR  with  NoVBS2  condition  had  no  basis  for  comparison.  The  results  (see 
Table  3)  imply  VBS2  AAR  capabilities  were  perceived  to  have  a  positive  impact  on  AARs.  Out 
of  the  30  responses  received,  only  10%  were  negative  (i.e.,  less  than  a  score  of  “3”),  compared 
with  70%  positive  (i.e.,  greater  than  “3”). 

Table  3 

Instructor  Ratings  for  Training  Outcome  Effects  of  AARs  with  VBS2 


How  much  do  you  agree  or  disagree  that: 

VBS2  playbacks  enhance  the  AAR  by  providing  a 
viewable  record  to  draw  teaching  points  from? 

VBS2  playbacks  during  AARs  promote  accountabil¬ 
ity  by  eliminating  uncertainty  about  who  did  what? 
VBS2  playbacks  during  AARs  make  it  easier  to 
focus  on  specific  mission  essential  tasks? 

VBS2  inspires  students  to  participate  in  the  AAR? 

Varying  the  camera  angles  of  the  VBS2  playback 
allows  for  better  visualization  of  the  battlefield? 
Alternating  friendly  and  enemy  Soldiers’  points  of 
view  fosters  discussion  of  teaching  points? 

The  value  of  performing  a  VBS2  mission  was  en¬ 
hanced  by  watching  the  playback  during  the  AAR? 

A  video  playback  of  the  VBS2  exercise  is  useful 
during  the  AAR? 

A  video  playback  from  the  VBS2  exercise  should  be 
a  standard  component  of  the  AAR? 

The  VBS2  AAR  Guide  is  a  useful  resource  for 
conducting  VBS2  enhanced  AARs? _ 


Instmctor  Instmctor  Instmctor 
1 _ 2 _ 3 

4  2  5 

4  3  5 

4  3  5 

4  3  5 

4  3  5 

4  3  4 

4  2  5 

4  2  5 

4  3  5 

4  4  4 


Mean  ( SD ) 
3.67  (1.53) 
4.00(1.00) 
4.00(1.00) 
4.00(1.00) 
4.00(1.00) 
3.67  (.58) 
3.67  (1.53) 
3.67  (1.53) 
4.00(1.00) 
4.00  (.00) 


Note.  The  rating  scale  ranged  from  1  (strongly  disagree)  to  5  (strongly  agree).  Figures  in  parentheses  are 
standard  deviations. 


Table  4  shows  instructors’  ratings  for  AARs  with  VBS2,  in  comparison  with  traditional 
AARs.  Once  again  the  results  were  favorable  for  AARs  using  VBS2  tools.  Of  the  24  responses 
received,  83%  were  favorable  for  AARs  with  VBS2  (i.e.,  greater  than  “3”),  while  no  ratings  were 
unfavorable  (i.e.,  less  than  “3”). 


17 


Table  4 


Instructor  Ratings  for  Learning  Impact  of AARs  with  VBS2 


Compared  to  traditional  AARs,  how  much  do  you 

agree  or  disagree  that: _ 

The  VBS2  playbacks  made  your  job  easier  as  the 
instructor  by  providing  a  viewable  record  of  events? 
The  free  camera  view  and  map  views  (2-D  and  3-D) 

provide  a  better  display  of  the  AO? _ 

The  VBS2  playback  better  enables  students  to 

visualize  the  AO  and  actions  during  an  AAR? _ 

VBS2  playback  provides  additional  detail  and 
clarity  of  events  that  occur  during  the  exercise? 

The  VBS2  playback  better  allows  students  to 
identify  any  deficiencies  which  may  have  occurred? 
The  VBS2  playback  better  allows  students  to 

understand  how  to  correct  deficiencies? _ 

The  VBS2  playback  better  allows  students  to  focus 
on  performance  of  specific  mission  essential  tasks? 
The  VBS2  playback  better  allows  students  to 
identify  strengths  and  actions  correctly  performed? 


Instructor  Instructor  Instructor 


Mean  (SD) 
4.00(1.00) 
4.00(1.00) 
4.00(1.00) 
4.33  (.58) 
4.00(1.00) 
4.33  (.58) 
4.33  (.58) 
4.33  (.58) 


Note.  The  rating  scale  ranged  from  1  (strongly  disagree)  to  5  (strongly  agree).  Figures  in  parentheses  are 
standard  deviations. 


Student  data 


Student  responses  also  supported  the  value  of  using  VBS2  AAR  capabilities  during 
AARs.  The  larger  samples  and  between-group  design  enabled  inferential  analysis  of  the  student 
data.  As  seen  in  Table  5,  students  in  the  AAR  with  VBS2  condition  produced  significantly 
higher  ratings  for  every  measure  of  training  outcomes  when  compared  to  the  AAR  with  no  VBS2 
condition.  Further,  all  of  the  effect  sizes  would  be  categorized  as  large  according  to  standards  set 
forth  by  Cohen  (1992). 

Students  in  the  experimental  condition  provided  head-to-head  ratings  for  AARs  with 
VBS2  versus  AARs  with  no  VBS2.  The  results  clearly  indicated  that  students  believed  the  use 
of  VBS2  tools  during  AARs  significantly  contributed  to  their  learning  experience  (see  Table  6). 
Of  the  263  responses  gathered  across  the  nine  questions,  only  2  (.76%)  indicated  a  negative 
rating  for  AARs  with  VBS2.  Nearly  80%  of  the  ratings  were  positive  for  AARs  withVBS2,  with 
40.7%  being  “agree”  and  38.4%  being  “strongly  agree.” 
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Table  5 


Average  Student  Ratings  for  AAR  Processes  and  Outcomes,  by  Research  Condition 


How  much  do  you  agree  or  disagree  that  the 

AAR  was  conducted  in  a  manner  that: 

AAR  with 
no  VBS2 
n  —  22 

AAR  with 
VBS2 

71  =  31 

^-value 

Effect 

Size 

P 

Leveraged  all  the  tools  to  help  me  learn? 

2.72  (.83) 

3.94 

(.73) 

5.63 

1.56 

.000 

Allowed  me  to  identify  any  deficiencies  which 
may  have  occurred? 

3.32(1.04) 

4.26 

(.68) 

3.97 

1.07 

.000 

Allowed  me  to  understand  how  to  correct  any 
deficiencies  which  may  have  occurred? 

3.23  (1.02) 

4.19 

(.60) 

4.33 

1.04 

.000 

Allowed  me  to  focus  on  performance  of  mission 
essential  tasks? 

3.36  (1.09) 

4.03 

(.70) 

2.71 

.73 

.009 

Allowed  me  to  identify  and  strengths  and 
actions  performed  correctly? 

3.23  (1.02) 

4.03 

(.71) 

3.40 

.91 

.001 

Inspired  me  to  participate? 

3.27  (.98) 

4.03 

(.75) 

3.19 

.87 

.002 

Note.  The  rating  scale  ranged  from  1  (strongly  disagree)  to  5  (strongly  agree).  Figures  in  parentheses  are 
standard  deviations. 


Table  6 

Average  Student  Ratings  for  Training  Impact  of  AARs  with  VBS2  (Experimental  Condition  Only) 


How  much  do  you  agree  or  disagree  that: 

The  value  of  performing  a  mission  in  VBS2  was  enhanced  by  watching  the 
video  playback  of  critical  events  during  the  AAR? 

A  video  playback  of  the  VBS2  exercise  is  useful  during  the  AAR? 

A  video  playback  from  the  VBS2  exercise  enables  me  to  visualize  the  AO  and 
actions  during  an  AAR? 

A  video  playback  from  the  VBS2  exercise  should  be  a  standard  component  of 
an  AAR  following  a  VBS2  training  mission? 

Compared  to  AARs  which  did  not  use  VBS2  playbacks,  I  believe  the  VBS2 
playback  better  allowed  me  to  identify  any  deficiencies  which  occurred? 
Compared  to  AARs  which  did  not  use  VBS2  playbacks,  I  believe  the  VBS2 
playback  better  allowed  me  to  understand  how  to  correct  deficiencies? 
Compared  to  AARs  which  did  not  use  VBS2  playbacks,  I  believe  the  VBS2 
playback  better  allowed  me  to  focus  on  performance  of  specific  tasks? 
Compared  to  AARs  which  did  not  use  VBS2  playbacks,  I  believe  the  VBS2 
playback  better  allowed  me  to  identify  strengths  &  actions  performed  correctly? 
Compared  to  AARs  which  did  not  use  VBS2  playbacks,  I  believe  the  VBS2 
playback  inspired  me  to  participate? _ 


n 

Mean  ( SD ) 

31 

4.38  (.61) 

31 

4.50  (.51) 

31 

4.34  (.60) 

31 

4.31  (.69) 

29 

4.00  (.84) 

29 

3.90  (.84) 

29 

3.87  (.86) 

26 

4.07  (.73) 

26 

4.00  (.92) 

Note.  The  rating  scale  ranged  from  1  (strongly  disagree)  to  5  (strongly  agree).  Figures  in  parentheses  are 
standard  deviations. 


Aggregated  across  participants  and  measures,  the  data  overwhelmingly  supported  the  use 
of  VBS2  AAR  capabilities  during  AARs.  Instructors  and  students  uniformly  rated  VBS2  AAR 
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features  as  having  a  positive  effect  on  learning  outcomes.  Observational  data  and  participant 
ratings  were  in  agreement,  indicating  that  students  participated  more  during  AARs  with  VBS2. 
Instructors  and  students  rated  AARs  with  VBS2  more  favorably  than  AARs  with  no  VBS2. 
Finally,  participants’  comments  reflected  positively  on  the  benefits  of  VBS2  AAR  capabilities. 

Impact  of  VBS2  AAR  Capabilities  on  Perceptions  of  Simulation-Based  Training 

Past  research  indicated  that  Soldier  “buy-in”  remains  a  significant  obstacle  to  the 
effective  use  of  GBT  (Topolski,  Leibrecht,  Cooley,  Rossi,  Lampton,  &  Knerr,  in  preparation). 
Instructors  and  students  alike  often  treat  simulations  as  a  game,  failing  to  take  the  learning 
objectives  seriously.  During  the  material  development  phase  of  this  research,  investigator 
observations  and  interview  feedback  from  instructors  and  course  managers  suggested  that  the 
VSB2  AAR  capabilities  were  currently  being  underutilized  during  AARs.  As  part  of  the 
experimental  investigation,  the  researchers  explored  the  impact  of  VBS2  AAR  capabilities  on 
Soldiers’  perceptions  of  simulation-based  training. 

Instructor  data 

Table  7  shows  instructor  ratings  for  computer-based  combat  simulations  after  completing 
an  AAR  using  VBS2  AAR  capabilities.  The  data  could  be  generally  described  as  neutral  to 
positive.  While  half  of  the  ratings  were  neutral,  the  remaining  half  were  positive. 

Table  7 

Instructor  Ratings  for  Effectiveness  of  Computer-Based  Combat  Simulations 


How  much  do  you  agree  or  disagree  that 
computer-based  combat  simulations  in  general: 

Instmctor 

1 

Instmctor 

2 

Instmctor 

3 

Mean  (SD) 

Are  an  effective  training  tool? 

3 

3 

4 

3.33  (.58) 

Build  students’  tactical  knowledge? 

3 

3 

4 

3.33  (.58) 

Increase  students’  procedural  knowledge? 

3 

4 

5 

4.00(1.00) 

Provide  valuable  lessons  that  students  can  apply 
on  the  job  to  improve  the  unit’s  performance? 

3 

4 

5 

4.00(1.00) 

Note.  The  rating  scale  ranged  from  1  (strongly  disagree)  to  5  (strongly  agree).  Figures  in  parentheses  are 
standard  deviations. 


Student  data 

Student  data  provided  stronger  support  for  the  positive  impact  of  VBS2  AAR 
capabilities  during  AARs  on  simulation-based  training.  Students  in  the  AAR  with  VBS2 
condition  rated  simulation-based  training  more  favorably  than  students  in  the  AAR  with  no 
VBS2  condition,  as  seen  in  Table  8.  Two  of  the  four  items  addressing  training  impact  yielded 
statistically  significant  differences  between  conditions,  while  the  other  two  produced  nearly 
significant  differences  (i.e.  ,P<  .10).  Further,  most  of  the  effect  sizes  were  moderate  to  large, 
indicating  that  using  VBS2  AAR  capabilities  during  the  AAR  had  a  substantial  impact  on  how 
students  viewed  simulation-based  training  overall. 
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Table  8 


Average  Student  Ratings  for  Effectiveness  of  Computer-Based  Combat  Simulations,  by  Research 
Condition 


How  much  do  you  agree  or  disagree  that 
computer-based  combat  simulations  in  general: 

AAR  with 
no  VBS2 
n  =  22 

AAR  with 
VBS2 
n  =  3 1 

^-value 

Effect 

Size 

P 

Are  effective  training  tools? 

Build  tactical  knowledge? 

Increase  procedural  knowledge? 

Provide  valuable  lessons  learned  that  I  can  apply 
on  the  job  to  improve  my  unit’s  performance? 

3.32(1.04) 

3.14(1.09) 

3.27(1.12) 

3.09(1.02) 

3.81  (.75) 
3.81  (.75) 
3.74  (.89) 

3.77  (.84) 

1.94 

2.97 

1.70 

2.66 

.54 

.72 

.46 

.73 

.058 

.004 

.09 

.01 

Note.  The  rating  scale  ranged  from  1  (strongly  disagree)  to  5  (strongly  agree).  Figures  in  parentheses  are 
standard  deviations. 


Students’  written  comments  corroborated  the  quantitative  data.  Students  in  the  AAR 
with  VBS2  condition  tended  to  provide  more  positive  comments  regarding  simulation-based 
training  than  students  in  the  AAR  with  no  VBS2  condition.  For  example,  comments  from 
students  in  the  AAR  with  VBS2  condition  highlighted  the  added  value  of  having  a  reviewable 
record  of  events  during  the  AAR:  (a)  “I  would  rate  the  AAR  playback  as  the  key  component 
which  makes  VBS2  a  valuable  training  tool,”  (b)  “I  think  this  part  of  the  program  is  the  most 
effective  part.  It  helps  identify  command  and  control  decisions  and  helps  leaders  be  able  to  see 
their  flaws  or  weak  points,”  and  (c)  “Very  effective  in  all  ways.  It  keeps  people  honest  on  how 
and  where  improvement  is  needed.”  In  comparison,  students  in  the  AAR  with  no  VBS2 
condition  had  less  positive  comments  regarding  simulations,  such  as  “This  system  is  a  waste  of 
time  and  money,”  and  “It’s  garbage.”  Other  comments  reflected  an  understanding  that  the 
system  could  be  utilized  more  effectively  by  instructors:  (a)  “We  haven’t  had  the  chance  to 
utilize  the  software  to  its  full  potential,”  and  (b)  “It  all  depends  on  how  Cadre  run  through 
simulations  -  at  other  facilities  I  have  had  Cadre  almost  treat  it  like  a  game  where  they  are 
competing  against  the  students  -  in  those  instances  nothing  was  learned.” 

Participants’  comments  indicated  a  need  for  more  VBS2  workstation  training  and  fewer 
technical  issues.  Representative  student  comments  included:  (a)  “If  we  had  more  train-up  things 
would  run  more  smoothly,”  (b)  “The  first  few  hours  of  running  a  mission  you  are  trying  to  learn 
the  controls  and  the  capabilities  more  than  accomplishing  the  mission,”  and  (c)  “We  were  never 
used  to  the  controls  and  the  system  always  locked  up.”  Instructors’  comments  were  typified  by 
“We’re  still  trying  to  figure  out  how  to  use  the  system.”  Members  of  the  research  team  noted 
that  VBS2  proficiency  and  use  were  very  uneven  across  instructors.  The  instructors  who  were 
less  proficient  operators  often  relied  on  the  more  seasoned  instructors  for  tips  and  guidance  on 
how  to  use  VBS2,  resulting  in  a  piecemeal  development  of  VBS2  capabilities. 

Finally,  participants  from  both  research  conditions  commented  that  technical  issues  were 
pervasive.  Example  comments  included:  (a)  “With  simulators  technical  problems  dominate  the 
training,”  (b)  “With  this  system,  sometimes  it’s  a  50/50  shot  of  the  network  working,”  and  (c) 
“We  waste  a  lot  of  time  getting  the  system  to  operate  correctly.”  These  comments  agreed  with 
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observations  made  by  the  research  team.  Technical  problems  were  not  uncommon  during  the 
execution  phase  of  the  VBS2  exercises,  but  occurred  less  frequently  during  the  AARs.  During 
the  execution  phase,  the  system  occasionally  locked  up  and  students  were  often  dropped  from  the 
simulation.  It  is  important  to  note  that  several  factors  likely  contributed  to  these  technical 
challenges,  including  software  and  hardware  limitations,  and  inexperienced  VBS2  operators. 

Conclusions  and  Recommendations 
Conclusions 

The  results  from  the  formative  evaluation  and  exploratory  investigation  indicate  that  the 
VBS2  AAR  Guide  is  an  effective  and  useful  tool  for  facilitators  as  part  of  simulation-based 
training  with  VBS2.  Instructor  and  SME  feedback  indicates  the  Guide  is  solidly  constructed, 
addresses  an  area  of  need,  and  is  a  welcomed  resource.  Observational  data,  instructor  and 
student  ratings,  and  instructor  and  student  comments  reveal  consistent  support  for  the  benefits  of 
using  the  VBS2  AAR  Guide  to  conduct  AARs  with  VBS2.  The  results  demonstrate  higher  ratings 
across  all  measures  for  AARs  conducted  using  the  VBS2  workstation  capabilities. 

The  VBS2  AAR  features  provide  a  reviewable  record  that  helps  units  discuss  collective 
or  individual  actions.  The  Guide  provides  a  general  framework  for  facilitators  to  leverage  the 
different  views  afforded  by  VBS2  to  illuminate  specific  troop  actions  and  critical  events.  The 
facilitators  who  participated  in  this  research  demonstrated  effective  control  of  the  content 
presented  to  students  by  using  bookmarks,  playbacks  and  playback  pauses,  free  camera  views, 
and  first-person  views.  They  employed  these  tools  to  review  the  commander’s  mission  and 
intent,  discuss  teaching  points,  review  TTP,  and  increase  student  accountability. 

Currently,  the  AAR  capabilities  of  VBS2  are  being  underutilized  by  facilitators  and 
trainers.  Interviews  with  instructors  and  observations  by  researchers  indicate  that  only  a  small 
fraction  of  facilitators  are  currently  using  the  VBS2  AAR  capabilities  during  AARs.  Instructor 
and  student  “buy-in”  for  the  value  of  simulation  training  remains  an  issue.  Evidence  from  this 
research  indicates  that  using  VBS2  AAR  capabilities  significantly  increases  Soldier  perceptions 
of  the  positive  value  of  simulation-based  training.  Using  the  VBS2  AAR  features  significantly 
increases  the  extent  to  which  Soldiers  take  “serious  games”  seriously.  Learning  outcomes  are 
likely  to  improve  if  more  facilitators  use  VBS2  AAR  capabilities  during  AARs.  The  VBS2  AAR 
Guide  is  a  valuable  resource  for  assisting  facilitators  with  this  effort. 

The  research  presented  here  is  limited  by  several  factors.  The  feedback  and  data  from 
instructors  in  the  formative  evaluation  and  exploratory  investigation  were  sparse,  limiting  the 
conclusions  that  can  be  drawn.  The  conclusions  are  also  limited  by  the  sample  of  instructors 
being  self-selected  rather  than  randomly  assigned.  That  is,  the  instructors  who  chose  to  conduct 
their  AAR  with  VBS2  were  likely  instructors  already  VBS2  savvy  and  holding  a  more  positive 
view  of  simulation-based  training. 

Finally,  several  limitations  are  associated  with  the  VBS2  AAR  Guide.  The  Guide  is 
limited  in  scope  because  it  is  designed  specifically  for  VBS2.  In  its  current  state,  it  is  not 
generalizable  beyond  AARs  following  simulation-based  training  missions  using  VBS2  as  the 
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simulation  platform.  Constructing  similar  guides  for  other  simulation  platforms  may  be 
warranted.  Potential  obstacles  for  instructors  using  the  Guide  may  include  increased  time 
demands  in  planning  and  preparing  for  AARs  and  time  lost  to  technical  difficulties. 


Lessons  Learned 

In  conducting  the  research  described  in  this  report,  the  investigators  identified  several 
noteworthy  lessons  learned. 

•  Employing  the  special  feedback  capabilities  of  GBT  software  to  improve  AARs  is  a 
training  multiplier  that  produces  practical  benefits  in  several  ways.  In  the  current 
research,  the  use  of  VBS2  AAR  tools  not  only  enhanced  the  AAR  process  and  outcomes 
but  also  increased  the  receptiveness  of  Soldiers  to  simulation-based  training.  However, 
the  value  of  using  the  special  AAR  capabilities  may  not  be  self-evident.  Without 
education  regarding  the  payoff  of  mastering  and  utilizing  GBT-based  AAR  tools,  trainers 
and  AAR  facilitators  may  not  realize  the  benefits  to  be  gained.  Thus,  deliberate  steps  are 
needed  to  inform  trainers  and  then  build  their  ability  to  use  the  new  tools.  Supporting 
packages  such  as  the  VBS2  AAR  Guide  can  directly  support  those  steps. 

•  Individual  differences  can  be  expected  among  facilitators  who  use  the  Guide.  While  the 
Guide  provides  a  general  framework  and  suggestions  on  how  to  use  the  VBS2  AAR 
capabilities,  trainers  exhibit  stylistic  differences  in  VBS2  use.  The  flexibility  of  optional 
views  allows  AAR  facilitators  to  choose  different  perspectives  for  specific  situations. 

For  example,  one  facilitator  may  choose  a  first-person  view  to  show  the  enemy's  position, 
while  another  might  use  the  enemy's  line  of  sight  to  indicate  their  location  and  intent. 
Variations  between  facilitators  likely  stem  from  individual  differences  in  openness  to 
technology,  instructional  styles,  etc.  The  differences  are  worthy  of  follow-up  research  to 
determine  which  views  or  other  feedback  features  are  the  most  effective  teaching  tools. 

•  Factors  in  the  training  environment  are  likely  to  hinder  efforts  to  promote  the  use  of 
GBT-based  AAR  tools.  Skepticism  about  simulation-based  training,  habits  acquired  as 
part  of  traditional  AARs,  and  unit  training  guidance  that  ignores  GBT-based  AAR 
techniques  can  work  against  employment  of  specialized  feedback  capabilities.  In  the 
absence  of  personal  evidence  of  the  benefits  of  using  simulation-specific  feedback  tools, 
trainers  may  well  find  it  hard  to  justify  expending  sufficient  time  and  effort  to  become 
proficient  with  the  tools.  Further,  the  scarcity  of  opportunities  and  resources  for  training 
on  GBT-based  AAR  tools  stands  in  the  way  of  trainers  who  want  to  become  GBT-savvy 
AAR  facilitators.  Obstacles  in  the  training  environment  should  be  taken  into  account 
when  planning  and  resourcing  simulation-based  training  programs.  These  programs 
would  do  well  to  include  AAR-specific  support  materials  such  as  the  VBS2  AAR  Guide. 

•  Clearly  defining  what  trainers  need  to  help  them  build  Soldiers’  TTP -based  competencies 
can  be  a  challenge.  In  the  current  project,  the  initial  concept  of  the  training  need — VBS2 
generated  videos  illustrating  high  priority  TTP — evolved  into  the  much  different  concept 
of  guidelines  for  conducting  VBS2-based  AARs.  This  evolution  was  driven  largely  by 
input  from  the  target  audience  regarding  what  schoolhouse  AAR  facilitators  really  need 
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to  help  them  exploit  VBS2’s  AAR  tools.  The  input  occurred  via  face-to-face  dialogue  in 
iterative  fashion,  which  was  critical  to  the  success  of  the  front-end  analysis.  Future 
research/development  teams  can  profit  by  investing  the  effort  to  carefully  define  the 
training  support  need  from  the  user’s  perspective. 

•  Involving  target  audience  representatives  in  the  development  efforts  is  critical  to  creating 
a  useful  training  support  product.  By  building  close  working  relationships  with  the  target 
audience  and  leveraging  their  input,  the  current  researchers  were  able  to  incorporate  user 
desires  and  preferences  throughout  the  development  process,  including  feedback-based 
revision  of  the  primary  product.  The  collaboration  between  researchers  and  users  was  an 
important  factor  in  the  project’s  success.  It  fostered  a  sense  of  ownership  among  the 
target  audience  and  brought  considerable  credibility  to  the  VBS2  AAR  Guide.  Building  a 
product  that  can  help  trainers  and  Soldiers  do  their  jobs  more  effectively  can  increase 
users’  willingness  to  adopt  the  product  as  a  valuable  resource. 

•  Creative  approaches  must  be  employed  to  collect  effective  data  from  institutional  and 
unit  trainers.  Trainers  in  both  arenas  are  often  heavily  burdened  with  multiple  duties  that 
compete  for  their  time.  Innovative  data  collection  strategies  must  be  developed  and 
implemented  to  (a)  identify  available  samples  and  (b)  incentivize  participants  to  provide 
quality  feedback.  For  example,  forming  relationships  with  military  leaders  and  trainers 
can  prove  invaluable  when  collecting  data.  Coordinating  a  data  collection  plan  with  these 
individuals  is  a  key  to  success.  Alternatively,  online  communities  of  practice  such  as  the 
MilGaming  forum  may  provide  an  avenue  for  data  collection.  In  that  approach,  posting  a 
product  on  a  website  for  review  can  yield  valuable  feedback  from  Army  developers  and 
practitioners. 


Recommendations 

Based  on  the  results  of  the  formative  evaluation  and  exploratory  investigation,  we  make 
the  following  recommendations  for  simulation-based  training: 

•  Increase  the  use  of  VBS2  AAR  capabilities:  While  numerous  organizations  are  using 
VBS2  exercises  for  training,  seemingly  few  facilitators  are  employing  the  VBS2  AAR 
features  during  their  AAR  sessions.  This  research  provides  clear  support  for  the  value  of 
using  VBS2  AAR  features.  Learning  outcomes  and  Soldiers’  acceptance  of  simulation- 
based  training  are  significantly  enhanced  through  the  use  of  VBS2  AAR  capabilities. 

•  Transition  the  VBS2  AAR  Guide  to  Army  units  and  institutions:  The  formative  evaluation 
indicates  that  the  Guide  is  well  constructed.  Evidence  from  the  exploratory  investigation 
suggests  the  Guide  is  an  effective  tool  for  leveraging  the  benefits  of  VBS2  AAR 
capabilities.  Transitioning  the  Guide  throughout  the  Army  could  accelerate  the  use  of 
VBS2  AAR  capabilities  and  help  maximize  the  benefits  of  simulation-based  training. 

•  Increase  Workstation  proficiency:  Proficiency  on  workstation  operations  is  critical  to 
successful  simulation-based  exercises.  A  VBS2  Workstation  consists  of  an  instructor  or 
lead  trainer  computer  that  is  acting  as  the  server  for  the  mission  being  conducted. 
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Proficiency  on  this  station,  which  includes  the  networking  and  AAR  functions  of  the 
system,  affords  trainers  greater  flexibility  to  impart  teaching  points  or  lessons  learned. 
Soldiers  are  less  inclined  to  use  and  value  systems  on  which  they  are  not  proficient. 
Substantial  train-up  is  required  for  the  training  audience  to  achieve  workstation 
proficiency,  and  the  train-up  must  be  enabled  by  Soldier- friendly  training  support 
materials.  Innovative  techniques  are  required  to  produce  acceptable  proficiency  in  the 
compressed  timeframe  typical  of  today’s  units  and  courses. 

•  Improve  Workstation  functioning:  The  U.S.  Army  has  made  a  substantial  investment  in 
VBS2  as  a  realistic,  low-cost  virtual  training  environment.  However,  the  workstations 
and  networks  often  fail  to  meet  the  demands  of  the  software  and  users.  Valuable  training 
time  is  lost  when  frequent  system  failures  occur,  and  Soldiers’  perceptions  of  simulation- 
based  training  suffer.  More  resources  should  be  allocated  for  mitigating  technical  issues 
through  better  equipment,  increased  technical  support,  and  improved  training. 

•  Conduct  additional  research  to  track  the  implementation/utilization  of  the  Guide:  The 
current  research  was  exploratory  in  nature.  Several  factors  limited  the  methodology  and 
data  collection.  As  such,  the  conclusions  are  limited  as  well.  Additional  research  may  be 
warranted  to  monitor  and  document  the  dissemination  and  utilization  of  the  Guide.  Such 
research  could  gather  operational  feedback  and  lessons  learned. 

•  Cross-reference  the  Guide  in  the  VBS2  Administrator  Manual.  The  Guide  has  been 
shown  to  be  a  valuable  resource  for  trainers  of  VBS2  exercises.  By  incorporating  a 
reference  to  the  Guide  in  the  VBS2  Administrator  Manual,  AAR  facilitators  can  be  made 
aware  of  the  Guide  and  its  positive  impact  on  training  outcomes. 

•  Designate  an  Army  proponent  for  the  VBS2  AAR  Guide :  Given  the  Guide’s  potential 
value  in  optimizing  the  Army’s  return  on  its  VBS2  investment,  it  may  be  worth  naming 
an  agency  as  the  cognizant  proponent  for  the  Guide.  The  role  of  the  proponent  agency 
would  be  to  manage  the  dissemination  and  maintenance  of  the  Guide  and  to  serve  as  an 
advocate  for  programmatic  and  resource  matters. 

•  Include  a  usage  guide  with  future  fielding  of  GBT  Systems:  Future  GBT  Systems  should 
include  a  usage  guide  that  preferably  precedes  the  release  of  the  software  to  the  field. 

This  would  empower  instructors  to  begin  strategizing  how  to  utilize  the  software  prior  to 
fielding. 

This  report  documents  the  creation  and  formative  evaluation  of  the  VBS2  AAR  Guide. 
The  Guide  was  developed  to  help  facilitators  optimize  the  effectiveness  of  AARs  of  VBS2 
missions.  It  is  a  valuable  addition  to  the  Soldiers’  Toolbox  for  Developing  TTP  (Topolski  et  al., 
2010).  Observational  data  suggests  that  there  is  a  clear  need  for  the  Guide.  The  results  of  the 
formative  evaluation  and  investigation  provide  converging  evidence  that  the  Guide  fulfills  that 
need.  The  results  also  strongly  suggest  that  effectively  employing  VBS2  AAR  capabilities 
during  an  AAR  is  a  cardinal  component  in  producing  “buy-in”  for  simulation-based  training 
among  facilitators  and  Soldiers.  Robust  dissemination  and  utilization  of  the  Guide  can  directly 
support  Army  efforts  to  capitalize  on  low-cost,  high- flexibility  simulation-based  training 
technology. 
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Appendix  A 

Acronyms  and  Abbreviations 

After  Action  Review 
Advanced  Leaders  Course 
Area  of  Operation 
Army  Reconnaissance  Course 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 

Basic  Officer  Leaders  Course 

Close  Combat  Tactical  Trainer 

Game-Based  Training 

Mean  (average) 

Massively  Multiple  Online  Role-Playing  Game 

Sample  Size 
Sub-Sample  Size 

Operational  Environment 

Probability  Level 
Personal  Computer 

Standard  Deviation 
Simulation  Networking 
Subject  Matter  Expert 
Standing  Operating  Procedures 

Tactical  Standing  Operating  Procedures 
Tactics,  Techniques,  and  Procedures 

Virtual  Battlespace-2 
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Appendix  B 


Data  Collection  Instruments 

Page 

Instructor  Feedback  Form . B-2 

AAR  Observation  Form . B-5 

Instructor  Profile  and  Survey . B-l  1 

Student  Profile  and  Survey . B-l 6 

“After  Action”  Protocol . B-l 9 
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INSTRUCTOR  FEEDBACK  FORM 


Instructor’s  Name : 


Date : 


Instructions 

Please  use  this  form  to  record  your  feedback  for  the  Suggested  Guidelines 
for  Integrating  a  VBS2  AAR  Playback  into  a  VBS2  Mission  AAR.  The 
research  team  will  collect  it  when  you  are  finished. 


1 .  Prior  to  conducting  your  AAR,  how  much  time  did  you  spend  on  reviewing  the 
guide? 


2.  Do  you  believe  the  time  you  spent  reviewing  the  Guide  was  sufficient  to  fully 
understand  the  contents  of  the  Guide? 


3.  What  is  the  most  useful  part  of  the  Guide ?  Why? 


4.  Which  parts  of  the  Guide  are  unclear  or  confusing?  Please  explain. 
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5.  Of  the  current  topics  covered  in  the  Guide,  please  indicate  which  if  any  needs 
to  be  expanded  upon.  (Check  all  that  apply) 

□  Purpose  of  the  Guide 

□  Why  use  the  VBS2  AAR  Playback 

□  Skills  Needed 

□  Integrating  VBS2-  Planning 

□  Integrating  VBS2-  Preparation 

□  Integrating  VBS2-  Conduct  the  AAR 

□  Appendix  A-  Capabilities 

□  Appendix  A-  Virtual,  Live,  Constructive 

□  Appendix  A-  Scenario  Development 

□  Appendix  A-  AAR 

□  Appendix  A-  Terrain  Types 

□  Appendix  A-  Training  and  Mission  Specific  Applications 

□  Appendix  A-  Limitations 

□  Appendix  A-  Mitigating  Limitations 


6.  Of  the  items  you  checked  in  question  5,  please  briefly  explain  what  needs  to  be 
expanded  upon  within  a  specific  section. 
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7.  Below  is  a  short  list  of  topics  which  could  be  added  to  the  Guide.  Please 

indicate  which  items  would  be  useful  to  add  to  the  Guide.  (Check  all  that  apply) 

□  Application  Scenario  (to  provide  an  example  of  who  would  benefit  from  the  guide) 

□  An  Appendix  outlining  a  sample  VBS2  AAR  which  matches  VBS2  capabilities  to  AAR 
events 

□  An  appendix  of  “hot  keys”  and  VBS2  functions 

□  Flow  chart  or  diagram  to  outline  the  VBS2  AAR  process 


8.  How  would  you  improve  the  VBS2  AAR  Instructor’s  Guide ?  (Consider 
organization,  contents,  and  packaging.) 


9.  Other  comments? 
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AAR  Observation  Form 


Parti:  EVENT  DETAILS 


Administrative  Data 

1 .  Observer: _ 

2.  Course  Name: _ 

3.  Location  (Installation  +  Room  #  or  Field  Site): _ 

4.  Date: _ 

5.  Exercise  Name: _ 

6.  #  of  instructors: _  #  of  students: _ 

7.  AAR  Condition  (circle  one):  Baseline  /  VBS2-Enhanced 

8.  Describe  the  training  site,  to  include  #  of  workstations;  sketch  a  diagram  of  the  site  below. 


9.  AAR  Start  Time:  _ 

10.  AAR  Stop  Time: 


Diagram  of  site  (sketch): 
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Part  II:  TIME-SAMPLING  CHRONOLOGY 


PARTICIPANT  ACTIVITIES 

IQ  —  Instructor  Queries 

IS  —  Instructor  Sketches  or  Wites 

IV  —  Instructor  Uses  VBS2  Functions 

SR  -  Student  Responds  to  VBS2 

ID  —  Instructor  Describes 

SQ-  Student  Queries 

SS  -  Student  Sketches  or  Wites 

IG—  Instructor  Guides  VBS2  Interpret’ n 

IF  -  Instructor  Fixes  VBS2  Problem 

SD  -  Student  Describes 

SI  -  Students  Interact 

IU  -  Instructor  Sets  Up  VBS2-AAR 

IE  —  Instructor  Errs  in  Operating  VBS2 

IA-  Instructor  Aborts  VBS2  Action 

VBS2  ACTIONS 

PN  —  Playback  No  Bookmark 

FP  —  Use  First  Person  View 

PP  —  Pause  Payback 

HL-UseHitUnes 

DM  —  Display  IVSssion  Briefing 

PB  —  Payback  via  Bookmark 

IVP  —  Use  Mouse  Arrow/Poi  nter 

BT  —  Use  Bookmark  Timestamp(s) 

SK-  Save  Key  Points  to  File 

FC  —  Use  Free  Camera  Views 

LO  —  Lock  Onto  Entity 

AT  —  Use  Audio  Track 

BN  —  Use  Bookmark  Notes 

OO  -  Other  (Describe) 

Block 

(Time) 

Activity 

Codes 

Description  of  instructor  and  student  behaviors,  interactions,  and  VBS2  operations 

VBS2  Action 
Codes 
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Part  II:  TIME-SAMPLING  CHRONOLOGY  (continued) 


PARTICIPANT  ACTIVITIES 

IQ  —  Instructor  Queries 

IS  —  Instructor  Sketches  or  Wites 

IV  —  Instructor  Uses  VBS2  Functions 

SR  -  Student  Responds  to  VBS2 

ID  —  Instructor  Describes 

SQ-  Student  Queries 

SS  -  Student  Sketches  or  Wites 

IG—  Instructor  Guides  VBS2  Interpret’ n 

IF  -  Instructor  Fixes  VBS2  Problem 

SD  -  Student  Describes 

SI  -  Students  Interact 

IU  -  Instructor  Sets  Up  VBS2-AAR 

IE  —  Instructor  Errs  in  Operating  VBS2 

IA-  Instructor  Aborts  VBS2  Action 

VBS2  ACTIONS 

PN  —  Playback  No  Bookmark 

FP  —  Use  First  Person  View 

PP  —  Pause  Payback 

BN  -  Use  Bookmark  Notes 

DM  —  Display  IVSssion  Briefing 

PB  —  Payback  via  Bookmark 

IVP  —  Use  Mouse  Arrow/Poi  nter 

BT  —  Use  Bookmark  Timestamp(s) 

SK-  Save  Key  Points  to  File 

FC  —  Use  Free  Camera  Viev\^ 

LO  -  Lock  Onto  Entity 

AT  —  Use  Audio  Track 

HL-UseHitUnes 

OO  -  Other  (Describe) 

Block 

(Time) 

Activity 

Codes 

Description  of  instructor  and  student  behaviors,  interactions,  and  VBS2  operations 

VBS2  Action 
Codes 
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Part  III:  SUPPLEMENTARY  INFORMATION 


1 .  What  level  of  VBS2-AAR  proficiency  did  the  instructor  exhibit?  (circle  one) 

Not  Observed  Little  Proficiency  Moderate  Proficiency  Strong  Proficiency  Expert  Proficiency 

2.  Following  ENDEX,  how  did  the  instructor  prepare  for  the  AAR? 


3.  At  the  beginning,  how  did  the  instructor  explain  the  role  of  the  VBS2  capabilities  in  the  AAR? 


4.  How  did  the  instructor  encourage  all  students  to  participate  in  the  AAR? 


5.  How  did  the  students  respond  to  the  VBS2  AAR  capabilities?  Consider  comments,  questions,  body 
language,  “ah-hah!”  reactions,  enthusiasm,  energy  level,  etc. 


6.  How  did  the  instructor  evaluate  student  performance? 

a.  _ Job  Aid  from  the  Instructor’s  P2P  Learning  Guide 

b.  _ Specialized  assessment  form  (obtain  a  blank  copy  if  possible) 

c.  _ Other  (describe): _ 

d.  _ Student  performance  was  not  evaluated 
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7.  How  much  did  the  instructor  employ  VBS2-AAR  capabilities  during  the  AAR?  (circle  one) 

Not  At  All  Once  or  Twice  Occasionally  (Missed  Chances)  Often  (Whenever  Appropriate)  Too  Much 
Comments: 


8.  What  evidence  did  you  see  that  the  instructor  used  ARI’s  VBS2  AAR  guidelines? 


9.  How  many  students  participated  fully  in  the  AAR?  (circle  one) 

None  One  About  One-Quarter  About  Half  About  Three-Quarters 
Comments: 


All  of  the  Students 


1 0.  How  did  the  instructor  depart  from  the  procedures  outlined  in  TC  25-20? 


1 1 .  Were  there  any  innovative  AAR  techniques  that  deserve  special  mention?  If  so,  describe  them. 
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INSTRUCTIONS  -  HOW  TO  USE  THIS  FORM 


This  form  is  to  be  used  by  a  VBS2-sawy  SME  to  record  observation  data  during  AARs 
conducted  as  part  of  VBS2  exercises  of  interest. 


STAGES: 

1 .  Advance  preparation:  Study  the  entire  form  before  the  scheduled  event  begins. 

2.  Parti:  Complete  items  1-8  as  the  AAR  is  getting  ready  to  start.  Record  actual  AAR  start  time  and 
stop  time  in  local  time  (military  format). 

3.  Part  II:  Record  chronology  data  continuously  using  the  table  provided. 

•  Break  the  data  recording  into  consecutive  3-minute  blocks. 

•  In  column  1 ,  record  the  start  time  of  each  block  (to  the  nearest  minute). 

•  In  column  2,  record  the  code  for  each  activity  observed  at  least  once  during  the  block. 

•  In  column  3,  describe  individual  behaviors,  interpersonal  interactions,  VBS2  operations,  and 
disruptions;  include  examples  of  activities.  When  in  doubt,  write  it  down! 

•  In  column  4,  record  the  code  for  each  VBS2  action  observed  at  least  once  during  the  block. 

4.  Part  III:  Complete  all  items  after  the  AAR  ends.  Glance  at  the  items  occasionally  during  the  AAR  to 
maintain  awareness.  Capture  as  much  information  as  possible. 

ACTIVITY  CODES  DEFINED: 

ID  -  Instructor  describes/explains/clarifies  mission,  actions,  events,  messages,  etc. 

SD  -  Student  describes/explains/clarifies  mission,  actions,  events,  messages,  etc. 

IQ  -  Instructor  queries  student(s)  to  explain  or  clarify  a  point  (pinpoint  or  open-ended  questions) 

SQ  -  Student  queries  instructor  to  obtain  explanation  or  clarification  of  a  point 

SI  -  Student  interacts  with  another  student  to  explain  a  point,  obtain  clarification,  etc. 

IS  -  Instructor  sketches  or  writes  on  white-board  or  butcher  paper 
SS  -  Student  sketches  or  writes  on  white-board  or  butcher  paper 
IU  -  Instructor  sets  up  VBS2  workstation  in  preparation  for  AAR  support 

IV  -  Instructor  uses  VBS2  function(s)  to  display,  highlight,  illustrate,  clarify,  answer  questions,  etc. 

IG  -  Instructor  guides  students  to  help  them  interpret  VBS2  displays  and  imagery 

SR  -  Student  responds  to  VBS2  display  by  means  of  comments,  questions,  body  language,  etc. 

IE  -  Instructor  errs  in  operating  VBS2,  identifying  a  problem,  or  representing  VBS2  information 
IF  -  Instructor  fixes  a  technical  problem,  including  troubleshooting  and  corrective  action 
IA  -  Instructor  aborts  a  VBS2  action  because  of  limited  proficiency  or  technical  problem 

VBS2  ACTION  CODES  DEFINED: 

DM  -  Display  Mission  Briefing  to  reiterate  the  training  objectives 

FC  -  Use  Free  Camera  view(s)  to  show  AO,  illustrate  a  teaching  point,  etc. 

PN  -  Playback  a  portion  of  the  exercise  using  no  bookmarks 

PB  -  Playback  a  portion  of  the  exercise  using  one  or  more  bookmarks 

LO  -  Lock  onto  a  vehicle  or  dismount  (friendly  or  enemy)  to  show  a  certain  perspective 

FP  -  Use  first-person  view  to  show  what  a  student  was  seeing 

MP  -  Use  mouse  arrow  or  pointer  device  to  highlight  key  elements  in  the  VBS2  display 

AT  -  Use  audio  track  during  playback  to  clarify  tactical  radio  communications 

PP  -  Pause  the  playback  while  a  critical  point  is  discussed 

BT  -  Use  bookmark  timestamp(s)  to  clarify  timing  of  key  event(s) 

BN  -  Use  bookmark  notes  to  organize  or  highlight  teaching  point(s) 

HL  -  Use  hit  lines  to  verify  shooter-target  identification  and  timing 
SK  -  Save  key  points  and/or  VBS2  materials  to  digital  file(s)  for  exporting 
00  -  Other  action  involving  VBS2  workstation  (describe) 
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INSTRUCTOR  PROFILE  AND  SURVEY 


Instructions:  The  questions  below  ask  for  your  opinions  about  VBS2  and  the  AAR  you  just 
completed.  Write-in  comments,  both  positive  and  negative,  are  encouraged.  Please  use  a  separate 
sheet  of  paper  if  you  need  additional  space. 


Rank  (current  or  retired): 

Branch/MOS: 

Time  on  active  duty:  yrs 

_  mos 

Unit: 

Time  as  Instructor  for  this  course _ 

yrs 

_ mos 

1.  Did  you  have  prior  instructor’s  experience  with  VBS2?  (Circle  one  and  explain.) 

O-None  1 -Little  Experience  2-Some  Experience  3-Much  Experience 


2.  Did  you  have  prior  experience  with  other  computer-based  combat  simulations  (e.g. 
DARWARS-AMBUSH)?  (Circle  one  and  list  which  ones.) 

O-None  1 -Little  Experience  2-Some  Experience  3 -Much  Experience 


3.  How  much  basic  VBS2  training  have  you  received?  (Circle  one  and  list  which  ones.) 

O-None  1 -Little  Experience  2-Some  Experience  3 -Much  Experience 


4.  How  much  VBS2  AAR  training  have  you  received?  (Circle  one  and  list  which  ones.) 

O-None  1 -Little  Experience  2-Some  Experience  3 -Much  Experience 


5.  How  much  experience  have  you  had  with  personal  computers?  (Circle  one  and  list 
which  ones.) 

O-None  1 -Little  Experience  2-Some  Experience  3 -Much  Experience 
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6.  How  much  do  you  agree  or  disagree  that  computer- 
based  combat  simulations  in  general: 

Circle  One  for  Each  Item 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

a.  are  an  effective  training  tool? 

1 

2 

3 

4 

5 

b.  build  students’  tactical  knowledge? 

1 

2 

3 

4 

5 

c.  increase  students’  procedural  knowledge? 

1 

2 

3 

4 

5 

d.  provide  valuable  lessons  that  students  can  apply  on  the  job  to 
improve  the  unit’s  performance? 

1 

2 

3 

4 

5 

Comments  and  Clarifications: 
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7.  How  much  do  you  agree  or  disagree  that: 

Circle  One  for  Each  Item 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

a.  VBS2  playbacks  enhance  the  AAR  by  providing  a  viewable 
record  to  draw  teaching  points  from? 

1 

2 

3 

4 

5 

b.  VBS2  playbacks  during  AARs  promote  accountability  by 
eliminating  uncertainty  about  who  did  what? 

1 

2 

3 

4 

5 

c.  VBS2  playbacks  during  AARs  make  it  easier  to  focus  on 
performance  of  specific  mission  essential  tasks? 

1 

2 

3 

4 

5 

d.  VBS2  inspires  students  to  participate  in  the  AAR? 

1 

2 

3 

4 

5 

e.  Varying  the  camera  angles  of  the  VBS2  playback  allows  for 
better  visualization  of  the  battlefield? 

1 

2 

3 

4 

5 

f.  Alternating  between  friendly  and  enemy  Soldiers’  point  of 
view  fosters  discussion  about  tactical  and  procedural  teaching 
points? 

1 

2 

3 

4 

5 

g.  The  value  of  performing  a  mission  in  VBS2  was  enhanced 
by  watching  the  video  playback  of  critical  events  during  the 
AAR? 

1 

2 

3 

4 

5 

h.  A  video  playback  of  the  VBS2  exercise  is  useful  during  the 
AAR? 

1 

2 

3 

4 

5 

i.  A  video  playback  from  the  VBS2  exercise  should  be  a 
standard  component  of  an  AAR  following  a  VBS2  training 
exercise? 

1 

2 

3 

4 

5 

j.  The  VBS2  AAR  Guide  is  a  useful  resource  for  conducting 
VBS2  enhanced  AARs? 

1 

2 

3 

4 

5 

Comments  and  Clarifications: 
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8.  Compared  to  traditional  AARs,  how  much  do  you 
agree  or  disagree  that: 

Circle  One  for  Each  Item 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

a.  the  VBS2  playbacks  made  your  job  easier  as  the  instructor  by 
providing  a  viewable  record  of  events? 

1 

2 

3 

4 

5 

b.  the  free  camera  view  and  map  views  (2-D  and  3-D)  provide  a 
better  display  of  the  AO? 

1 

2 

3 

4 

5 

c.  the  VBS2  playback  better  enables  students  to  visualize 
the  AO  and  actions  during  an  AAR? 

1 

2 

3 

4 

5 

d.  VBS2  playback  provides  additional  detail  and  clarity  of  the 
events  that  occurred  during  the  exercise? 

1 

2 

3 

4 

5 

e.  the  VBS2  playback  better  allows  students  to  identify  any 
deficiencies  which  may  have  occurred? 

1 

2 

3 

4 

5 

f.  the  VBS2  playback  better  allows  students  to  understand  how 
to  correct  any  deficiencies  which  may  have  occurred? 

1 

2 

3 

4 

5 

g.  the  VBS2  playback  better  allows  students  to  focus  on 
performance  of  specific  mission  essential  tasks? 

1 

2 

3 

4 

5 

h.  the  VBS2  playback  better  allows  students  to  identify  any 
strengths  and  actions  correctly  performed? 

1 

2 

3 

4 

5 

Comments  and  Clarifications: 
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9.  How  much  do  you  agree  or  disagree  that  the  VBS2 
AAR  Guide: 

Circle  One  for  Each  Item 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

a.  is  a  useful  aid  for  incorporating  VBS2  tools  into  an  AAR? 

1 

2 

3 

4 

5 

b.  is  clearly  written? 

1 

2 

3 

4 

5 

c.  contains  the  right  level  of  detail  for  facilitators? 

1 

2 

3 

4 

5 

d.  avoids  unnecessary  or  excess  information? 

1 

2 

3 

4 

5 

e.  is  well  organized  and  easy  to  follow? 

1 

2 

3 

4 

5 

f.  is  a  valuable  addition  to  conduct  VBS2  exercises? 

1 

2 

3 

4 

5 

g.  can  help  facilitators  achieve  learning  outcomes? 

1 

2 

3 

4 

5 

h.  uses  language  familiar  to  facilitators? 

1 

2 

3 

4 

5 

Comments  and  Clarifications: 

Other  Comments: 
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STUDENT  PROFILE  AND  SURVEY 


Instructions:  The  questions  below  ask  for  your  opinions  about  the  AAR  you  just  completed.  Write-in 
comments,  both  positive  and  negative,  are  encouraged.  Please  use  a  separate  sheet  of  paper  if  you 
need  additional  space. 


Rank: _  Branch/MOS: _ 

Time  in  Service: _ yrs _ mos  Unit: _ 

1.  Did  you  have  prior  experience  with  VBS2  before  this  course?  (Circle  one  and  explain.) 

O-None  1 -Little  Experience  2-Some  Experience  3-Much  Experience 


2.  Did  you  have  prior  experience  with  other  computer-based  combat  simulations  (e.g., 
DARWARS  AMBUSH)?  (Circle  one  and  list  which  ones.) 

O-None  1 -Little  Experience  2-Some  Experience  3 -Much  Experience 


3.  How  much  do  you  agree  or  disagree  that  computer- 
based  combat  simulations  in  general: 

Circle  One  for  Each  Item 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

a.  are  an  effective  training  tool? 

1 

2 

3 

4 

5 

b.  build  tactical  knowledge? 

1 

2 

3 

4 

5 

c.  increase  procedural  knowledge? 

1 

2 

3 

4 

5 

d.  provide  valuable  lessons  that  I  can  apply  on  the  job  to 
improve  my  unit’s  performance? 

1 

2 

3 

4 

5 

Comments  and  Clarifications: 
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4.  How  much  do  you  agree  or  disagree  that  the  AAR 
was  conducted  in  a  manner  that: 

Circle  One  for  Each  Item 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

a.  Leveraged  all  the  tools  available  to  help  me  learn? 

1 

2 

3 

4 

5 

b.  Allowed  me  to  identify  any  deficiencies  which  may  have 
occurred? 

1 

2 

3 

4 

5 

c.  Allowed  me  to  understand  how  to  correct  any  deficiencies 
which  may  have  occurred? 

1 

2 

3 

4 

5 

d.  Allowed  me  to  focus  on  performance  of  specific  mission 
essential  tasks? 

1 

2 

3 

4 

5 

e.  Allowed  me  to  identify  any  strengths  and  actions  performed 
correctly? 

1 

2 

3 

4 

5 

f.  Inspired  me  to  participate? 

1 

2 

3 

4 

5 

Comments  and  Clarifications: 

IF  VBS2  WAS  NOT  USED  DURING  YOUR  AAR,  THEN  STOP  HERE. 

IF  VBS2  WAS  USED  DURING  YOUR  AAR,  THEN  PROCEED  TO  THE  FOLLOWING 
QUESTIONS. 
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5.  How  much  do  you  agree  or  disagree  that: 

Circle  One  for  Each  Item 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

a.  The  value  of  performing  a  mission  in  VBS2  was  enhanced  by 
watching  the  video  playback  of  critical  events  during  the  AAR? 

1 

2 

3 

4 

5 

b.  A  video  playback  of  the  VBS2  exercise  is  useful  during  the 
AAR? 

1 

2 

3 

4 

5 

c.  A  video  playback  from  the  VBS2  exercise  enables  me  to 
visualize  the  AO  and  actions  during  an  AAR? 

1 

2 

3 

4 

5 

d.  A  video  playback  from  the  VBS2  exercise  should  be  a 
standard  component  of  an  AAR  following  a  VBS2  training 
mission? 

1 

2 

3 

4 

5 

e.  Compared  to  AARs  which  did  not  use  VBS2  playbacks,  I 
believe  the  VBS2  playback  better  allowed  me  to  identify  any 
deficiencies  which  may  have  occurred? 

1 

2 

3 

4 

5 

f.  Compared  to  AARs  which  did  not  use  VBS2  playbacks,  I 
believe  the  VBS2  playback  better  allowed  me  to  understand 
how  to  correct  any  deficiencies  which  may  have  occurred? 

1 

2 

3 

4 

5 

g.  Compared  to  AARs  which  did  not  use  VBS2  playbacks,  I 
believe  the  VBS2  playback  better  allowed  me  to  focus  on 
performance  of  specific  mission  essential  tasks? 

1 

2 

3 

4 

5 

h.  Compared  to  AARs  which  did  not  use  VBS2  playbacks,  I 
believe  the  VBS2  playback  better  allowed  me  to  identify  any 
strengths  and  actions  performed  correctly? 

1 

2 

3 

4 

5 

i.  Compared  to  AARs  which  did  not  use  VBS2  playbacks,  I 
believe  the  VBS2  playback  inspired  me  to  participate? 

1 

2 

3 

4 

5 

Comments  and  Clarifications: 

Other  Comments: 
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HOTWASH  PROTOCOL 


Notes  for  the  Hotwash  Facilitator 

Purpose:  To  feedback  from  Instructors  regarding  the  VBS2  AAR  Guide  and  conducting  an 
AAR  withVBS2 

Time:  There  is  a  30-minute  time  limit  on  the  hotwash 

Data  Recording:  Take  written  notes  and  tape  record  all  comments 

Data  Transfer:  Forward  all  notes  and  tape  recordings  to  the  Lead  Scientist  ASAP _ 

1)  Did  you  have  enough  time  available  to  study  the  guide  in  advance?  How  much  time 
would  you  need? 


2)  How  does  the  AAR  with  VBS2  differ  from  what  you  typically  do  during  an  AAR? 


3)  Is  performing  the  AAR  with  VBS2  worth  the  time  and  effort? 


4)  What  is  the  biggest  benefit  from  using  VBS2  playback  during  an  AAR? 


5)  What  is  the  biggest  drawback  of  using  VBS2  playback  during  an  AAR? 
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6)  Do  you  have  any  comments  you  would  like  to  share  about  the  bookmarking  or  timestamp 
tools  of  VBS2? 


7)  Which  VBS2  AAR  tools  did/would  you  choose  not  to  use?  Why? 


8)  Is  there  a  feature  you  would  like  to  see  added  to  the  VBS2  specifically  for  AARs? 


9)  What  concerns  do  you  have  about  the  VBS2  AAR  Guide? 


10)  What  concerns  do  you  have  about  conducting  an  AAR  with  VBS2? 


1 1)  Any  other  comments  or  thoughts  you  would  like  to  share? 
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Appendix  C 
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/  \ 

VI RTUAL  BATTLESPACE  2  (VBS2) 

AFTER  ACTI  ON  REVI EW  (AAR)  GUI  DE 

V _ _ _ _ _ J 

♦  ♦  A  Training  Support  Tool  ♦  ♦ 


About  This  Guide 

This  tool  gives  AAR  facilitators  a  set  of  guidelines  for  conducting  AARs  of 
VBS2  missions.  By  exploiting  VBS2's  special  AAR  capabilities,  the  Guide  is 
designed  to  enhance  learning  for  the  training  audience. 


Developed  by: 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 

Fort  Hood,  Texas 


November  2010 
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What’s  Inside  ... 


What  is  VBS2? 

1 

What  Are  VBS2's  AAR  Tools? 

1 

Why  Use  the  VBS2  AAR  Tools? 

2 

What  VBS2  Workstation  Skills  Are  Needed? 

2 

How  to  Set  Up  the  VBS2  Site? 

2 

Guidelines  for  Integrating  VBS2  into  the  AAR 

2 

Planning  the  AAR 

2 

Preparing  for  the  AAR 

3 

Conducting  the  AAR 

3 

Basic  Capabilities  and  Limitations  of  VBS2 

A- 1 

AAR  Capabilities  of  the  VBS2  Workstation 

B-l 

Sources  of  Additional  Information 

C-l 

What  is  VBS2? 


♦  VBS2  is  a  first-person  shooter,  semi-immersive,  virtual  training  simulation. 

♦  The  multi-player  simulation  runs  on  a  network  of  PC-based  workstations. 

♦  VBS2’s  virtual  environment  enables  players  to  operate  and  interact  realistically. 

♦  A  VBS2-proficient  trainer  controls  exercises,  serving  as  the  system  administrator. 

♦  VBS2  provides  special  AAR  tools  that  can  enhance  the  feedback  process. 

♦  See  “Basic  Capabilities  and  Limitations  of  VBS2”  below  for  more  information  on 
VBS2. 


What  are  VBS2’s  AAR  tools? 


♦  Visualization  tools  such  as  terrain  maps,  versatile  area  of  operations  (AO)  views, 
and  hit  lines. 

♦  Playback  features  that  recreate  visual  and  audio  dimensions  of  the  mission. 

♦  Teaching  point  tools  such  as  mission  briefing,  bookmarking,  and  notes. 

♦  Timestamp  features  that  help  verify  timing  and  chronological  sequence. 

♦  Take-home  tools  such  as  saving  mission  segments  and  editing/exporting  files. 
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♦  See  “AAR  Capabilities  of  the  VBS2  Workstation”  below  for  information  on  when  to 
use  each  tool. 


Why  use  the  VBS2  AAR  tools? 


♦  Eliminate  uncertainty  about  who  did  what. 

♦  Make  the  AARs  more  interesting. 

♦  Optimize  Soldier  participation. 

♦  Sharpen  battlefield  visualization  skills. 

♦  Strengthen  the  learning  process. 

♦  Enhance  the  effectiveness  of  the  AARs. 

♦  Improve  the  learning  outcomes. 


What  VBS2  workstation  skills  are  needed? 


♦  Ability  to  boot  up  and  initialize  the  workstations  in  the  proper  sequence. 

♦  Basic  proficiency  as  a  VBS2  operator,  based  on  training  and  practice. 

♦  Training  and  practice  in  employing  the  AAR  capabilities  of  the  workstation. 

♦  Ability  to  troubleshoot  and  fix  problems  with  the  workstation  and  large  display. 

♦  See  the  summary  of  VBS2  capabilities  in  “Basic  Capabilities  and  Limitations  of 
VBS2”  below. 


How  to  set  up  the  VBS2  site? 


♦  Boot  up  and  initialize  the  workstations  in  the  proper  sequence. 

♦  Connect  the  controller’s  workstation  to  the  large  screen  display  and  test  it. 

♦  Select  the  desired  tactical  communication  settings  to  match  the  desired  network. 

♦  Conduct  an  operational  check  to  make  sure  the  VBS2  functions  are  working. 

♦  Verify  that  voice-over-internet-protocol  (VOIP)  and  interconnectivity  are  working. 


Guidelines  for  integrating  VBS2  into  the  AAR 


Planning  the  AAR 


♦  Use  visualization  capabilities  to  help  formulate  the  AAR  strategy. 

•  Use  the  Training  Mission  menu  option  to  review  the  training  objectives. 
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•  Use  free  camera  mode  to  survey  the  AO,  key  events,  likely  mistakes,  etc. 

•  Conduct  virtual  recon  to  identify  key  phases  of  the  mission. 

•  Study  the  AO  with  various  views  to  anticipate  friendly  and  enemy  actions. 

♦  Use  map  capabilities  to  select  suitable  AAR  terrain. 

•  Use  2-D  map  views  to  become  familiar  with  AO  terrain. 

•  Survey  3-D  map  to  identify  candidate  AAR  sites. 

•  Use  default  camera  to  decide  which  terrain  will  best  support  AAR  discussion. 

♦  Review  VBS2  workstation  tools  to  choose  an  optimal  mix  for  the  AAR. 

•  Using  the  section  below  titled  “AAR  Capabilities  of  the  VBS2  Workstation,” 
decide  which  VBS2  AAR  tools  best  support  the  training. 

•  Build  specific  tools  (alone  or  in  combination)  into  the  AAR  plan,  by  stage. 

♦  Schedule  adequate  time  to  work  on  a  VBS2  workstation  during  AAR  planning. 


Preparing  for  the  AAR 

♦  Update  own  proficiency  as  a  VBS2  administrator  beforehand. 

•  Review  the  list  of  VBS2  workstation  skills  needed  (above,  see  page  2). 

•  Assess  own  proficiency  with  VBS2  AAR  tools  called  for  in  the  AAR  plan. 

•  Practice  on  a  workstation  to  reach  proficiency  on  stumbling-block  skills. 

♦  Dry-run  the  AAR  before  the  exercise. 

•  Step  through  the  AAR  plan,  using  the  VBS2  workstation  as  needed. 

•  Using  the  free  camera  mode,  note  likely  points  for  bookmarking. 

•  Position  the  default  camera  to  cover  critical  tasks  and/or  likely  mistakes. 

•  Identify  mission  segments  or  other  items  to  be  saved  for  use  after  the  AAR. 

•  Finalize  the  AAR  plan  based  on  the  dry-run. 

♦  Manage  AAR  recording  during  mission  execution. 

•  Start  the  AAR  recording  function  at  the  beginning  of  mission  execution. 

•  Pause  AAR  recording  during  administrative  breaks,  discussions,  etc. 

•  Use  the  bookmark  feature  to  designate  key  events  for  AAR  playback. 

•  Use  bookmark  notes  to  save  key  observations  for  AAR  discussion. 

•  Maintain  a  list  of  bookmarks  to  use  as  a  reminder  during  the  AAR. 

♦  Ready  the  controller’s  workstation  and  AAR  materials  after  the  mission  ends. 

•  Review  bookmarks  to  help  organize  AAR  discussion  around  teaching  points. 

•  Select  bookmarked  events  to  support  AAR  discussion. 

•  Enter  additional  bookmark  notes  to  guide  discussion  of  teaching  points. 

•  Verify  the  controller’s  workstation  is  connected  to  the  large  screen  display. 

♦  Schedule  adequate  time  on  a  VBS2  workstation  during  AAR  preparation. 
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Conducting  the  AAR 


♦  Set  the  AAR  stage  for  the  participants. 

•  Display  the  Training  Mission  to  focus  participants  on  the  training  objectives. 

•  Explain  how  the  VBS2  system  is  being  used  to  enhance  the  AAR. 

•  Encourage  close  attention  to  VBS2’s  video  and  audio  information. 

•  Use  map  views  or  free  camera  “fly-over”  to  establish  common  picture  of  AO. 

♦  Maintain  the  AAR  focus  on  teaching  points. 

•  Use  video/audio  playback  to  highlight  or  clarify  key  teaching  points. 

•  Use  playback  and  timestamps  to  verify  the  chronological  order  of  events. 

•  Use  free  camera  views  (e.g.,  Figure.  1)  to  illustrate  different  perspectives. 

•  Use  first-person  views  to  bring  specific  participants  into  the  discussion. 

•  Display  selected  bookmark  notes  to  support  teaching  points  and  discussion. 

•  Re-display  the  Training  Mission  to  re-focus  on  the  training  objectives. 

•  Explain  what  Soldiers  are  seeing  (e.g.,  third-person  view,  hit  lines)  as  needed. 

♦  Save  AAR  materials  for  later  use. 

•  In  accordance  with  the  AAR  plan,  save  VBS2  take-home  materials. 

•  Save  additional  VBS2  materials  as  identified  during  the  mission  or  the  AAR. 

•  Insert  a  list  of  saved  files  into  the  hand-written  notes,  with  cross-references. 

Figure  1 .  Illustrative  free  camera  view  provided  by  VBS2  (edited  with  text  and  arrows). 
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Basic  Capabilities  and  Limitations  of  VBS2 


Capabilities 

VBS2  is  a  PC-based,  fully  interactive,  three  dimensional,  first  person  shooter,  synthetic 
environment  capable  of  simulating  a  wide  range  of  situations  at  the  Company  level  and 
below.  Users  execute  missions  in  a  semi-immersive  virtual  environment  that  accurately 
represents  the  OE.  The  system  interoperates  within  a  Live,  Virtual,  Constructive  (LVC) 
environment  and  interfaces  with  U.S.  Army  and  Joint  Command  and  Control  (C2) 
systems.  VBS2  accurately  emulates  most  U.S.  Army  weapon  systems  and  the  effects  of 
those  weapons,  both  mounted  and  dismounted.  Tactical  radio  communication  (live  radio 
traffic)  is  modeled  through  the  use  of  headsets  with  microphones  using  (VOIP). 

Live-Virtual-Constructive  Environment 

VBS2  can  be  federated  with  other  High  Level  Architecture  (HLA)  compliant  simulations 
to  meet  specific  training  outcomes.  For  example,  dismounted  infantry  in  VBS2  can  be 
connected  with  a  high  fidelity  armored  vehicle  simulator.  One  can  also  simulate  a 
Special  Forces  team  conducting  counter-insurgency  missions,  while  the  overall 
campaign  is  controlled  by  a  higher  level  constructive  simulation  such  as  OneSAF. 

Scenario  Development 

Comprehensive  scenarios  can  be  created  through  the  use  of  a  Virtual  Training  Kit  and 
edited  while  the  scenario  is  running  through  the  use  of  a  Real  Time  Editor  (RTE).  Users 
may  place  the  scenario  in  pause  mode  to  conduct  in-stride  AARs  and  either  restart  the 
mission  where  it  was  stopped  or  reset  the  mission  completely.  Scenarios  may  be 
customized  to  the  user’s  specific  equipment  and  mission  needs.  A  built-in  tutorial  can 
show  players  the  control  features  and  enable  them  to  practice  executing  tasks. 

AAR  Tools 

In  addition  to  the  RTE,  VBS2  includes  robust  AAR  tools  allowing  complete  review  of 
mission  execution.  Among  the  tools  are  playback  of  mission  execution  with  radio  traffic, 
and  free  camera  views  that  allow  the  facilitator  to  review  the  actions  from  several  angles 
during  the  playback.  A  bookmark  feature  is  available  that  enables  the  administrator  to 
mark  a  critical  stage  of  the  mission  and  quickly  go  back  to  it  during  the  AAR. 

The  AAR  tools  may  also  be  utilized  to  produce  videos  for  communicating  TTP.  The  TTP 
are  executed  while  VBS2  captures  the  mission  with  the  video/audio  recording  functions. 
The  AAR  file  can  then  be  edited  for  use  as  a  communication  tool  and  presented  to 
Soldiers. 
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Terrain  Types 

VBS2’s  terrain  database  can  replicate  numerous  geo-typical  and  geo-referenced  terrain 
models  around  the  world.  Some  of  these  terrain  models  are  listed  below. 


Afghanistan 

Geo-Typical 

Geo- Referenced 

Fictional 

Kandahar 

Tarin  Kowt 

T  ropical 

SW  USA 

Eastern  Europe 

Shakarat  Village 
(Iraq) 

National  Training 
Center 

As  Samawah  (Iraq) 
Baghdad-Green  Zone 

Sahrani  (multiple  terrain 
types) 

Porto  (island  with  small 
town) 

Rahmadi  (small  island) 

Training  and  Mission-Specific  Applications 

VBS2  can  be  used  to  train  individual  and  collective  tasks  in  support  of  full  spectrum 
operations.  Combined  Arms  and  Joint  Operations  missions  can  also  be  trained  using 
VBS2.  Some  examples  of  specific  training  and  mission-based  applications  are  listed 
below. 

•  Mission  rehearsal  and/or  AO  familiarization, 

•  Tactical  training, 

•  Crew/section  battle  drills, 

•  C2  training  at  the  Company  level  and  below, 

•  Combined  Arms  or  Joint  Training, 

•  Individual  training, 

•  Collective  training  up  to  Company  level, 

•  Convoy  training  (including  integration  of  virtual  reality  technology), 

•  Improvised  explosive  device  (IED)  defeat, 

•  Analysis  of  options  (decision  support), 

•  Fire  support  /  forward  air  controller  training, 

•  Complementary  virtual  environment  for  live  and  constructive  simulation  or  crew 
trainers, 

•  Navigation, 

•  Mission  simulation  (e.g.,  aviation  elements  practicing  landing  zone  (LZ) 
procedures), 

•  Vehicle  checkpoints  and  area  control, 

•  Procedural  training  for  unmanned  aerial  vehicle  (UAV)  operators, 

•  Cultural  awareness  training, 

•  Visualization  of  weapon  effects, 
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•  Weapon  (or  platform)  familiarization  or  experimentation, 

•  Training  in  urban  environments  (e.g.,  military  operations  on  urbanized  terrain 
(MOUT)  battle  drills),  and 

•  Contact  drills. 

Limitations 

•  Limited  capability  to  train  above  the  Company  level.  Since  VBS2  is  a  first-person 
simulation,  information  sent  to  higher  echelons  must  come  from  leaders 
executing  the  game.  First-person  simulations  are  video  games  characterized  by 
a  three-dimensional  view  from  the  player's  perspective,  often  holding  a  weapon 
in  front.  Each  position  must  have  an  active  player  in  the  game  in  order  to 
generate  combat  information.  Any  information  sent  to  higher  echelons  must 
come  from  these  players.  VBS2  will  not  generate  this  information  on  its  own. 
Therefore  training  at  the  battalion  level  is  dependent  upon  having  enough  units  in 
the  game  to  accurately  represent  the  combat  reporting  sent  to  battalion  and 
higher  headquarters.  VBS2  cannot  generate  this  type  of  information  without  input 
from  player  units. 

•  Limited  ability  to  train  hands-on  tasks.  For  example,  the  game  can  effectively 
train  communi-cation  within  a  tank  crew  but  is  not  as  effective  at  training  gunnery 
crew  drills.  The  desktop  nature  of  the  game  limits  its  ability  to  replicate  actual 
crew  stations.  Some  movements  or  actions  of  the  crew  are  artificial  and  do  not 
accurately  represent  actual  movements. 

•  Limited  system  fidelity  to  capture  all  movements  and  objects  during  AAR 
recording.  The  AAR  does  not  capture  every  frame  of  action  for  playback.  For 
example,  a  player  can  open  vehicle  doors  and  trunks  in  game  but  the  AAR  may 
not  fully  capture  these  actions  for  playback.  The  doors  will  either  not  open  at  all 
during  playback  or  will  only  partially  open.  Another  system  fidelity  limitation  is  the 
inability  of  the  AAR  to  replicate  objects  that  are  replaced  in  the  player’s  weapons 
inventory.  For  example,  if  a  player  is  equipped  with  a  search  mirror,  the  mirror 
replaces  the  sidearm  in  the  player’s  inventory.  This  mirror  can  be  used  and 
viewed  by  all  players  in  game  during  AAR  recording.  However,  the  AAR  will  not 
replicate  the  mirror  during  playback,  showing  only  the  sidearm  that  is  present  in 
the  player’s  inventory.  These  limitations  may  be  mitigated  by  using  video 
software  such  as  FRAPS  to  record  a  player’s  perspective  directly  from  their 
computer  screen. 

•  Limited  ability  to  train  full  tactical  communication.  VBS2’s  desktop  nature 
restricts  its  ability  to  replicate  actual  tactical  communication  between  echelons 
over  distance.  Use  of  headphones  with  microphones  can  mitigate  this  limitation, 
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but  players  may  still  be  tempted  to  talk  directly  to  another  unit  rather  than  use  the 
VOIP  mechanism.  Separating  sections  by  means  of  partitions  or  different  rooms 
also  helps  mitigate  this  limitation. 

Mitigating  Limitations 

Leaders  can  mitigate  system  limitations  by  carefully  analyzing  the  training  event  to 
determine  appropriate  tasks  for  execution  in  VBS2  simulation.  Leaders  should 
determine  the  tasks  that  best  leverage  VBS2  capabilities  and  develop  scenarios 
accordingly.  For  those  tasks  not  well  suited  for  VBS2,  other  simulations  such  as  the 
Close  Combat  Tactical  Trainer  (CCTT)  may  be  an  option  to  exercise  newly  developed 
TTP. 


AAR  Capabilities  of  the  VBS2  Workstation 


The  VBS2  software’s  primary  AAR  support  features  are  outlined  in  the  table  below. 


VBS2  Feature 

When  to  Use 

AAR  Recording 

•  At  the  beginning  of  mission  execution,  start  the  AAR  recording 
function. 

•  Position  default  camera  for  optimal  coverage  during  set-up  of 
the  controller’s  workstation. 

Recording  Pause 

•  Temporarily  suspend  AAR  recording  during  breaks  in  mission 
execution. 

Bookmark 

•  Mark  a  critical  point  during  mission  execution  to  serve  as  a 
later  reference  or  index. 

•  Quickly  jump  to  a  critical  point  during  playback  in  AAR,  using  a 
bookmark  recorded  earlier. 

Bookmark  Notes 

•  Type  in  observations  and  comments  during  mission  execution, 
to  serve  as  reminder  later. 

•  Review  notes  during  AAR  preparation,  to  select  and  organize 
events  and  teaching  points. 

•  Display  selected  notes  while  conducting  the  AAR,  to  support 
teaching  points  &  discussion. 

Training  Mission 

•  Display  mission  briefing  at  the  AAR’s  start,  to  focus  participants 
on  training  objectives. 

•  Display  mission  briefing  later  in  the  AAR,  to  restore  focus  on 
specific  training  objectives. 

Map  View  (2-D,  3- 
D) 

•  Study  the  AO  in  advance  of  scheduled  training,  to  support  AAR 
planning  and  preparation. 

•  Display  an  AO  map  at  the  start  of  the  AAR,  to  establish  a  big 
picture  for  the  participants. 

•  Display  an  AO  map  during  the  AAR,  to  support  teaching  points 
or  respond  to  questions. 

C-10 


Free  Camera  View 

•  Recon  the  AO  during  exercise  preparation,  to  select  an  optimal 
default  camera  position. 

•  Display  key  views  at  the  start  of  the  AAR,  to  highlight  critical 
areas  or  features  of  the  AO. 

•  Compare  the  friendly  perspective  with  the  enemy  perspective 
(as  needed)  during  the  AAR. 

•  Display  key  views  during  the  AAR,  to  support  teaching  points 
or  respond  to  questions. 

Playback 

•  Replay  bookmarked  portions  of  the  mission  during  AAR,  to 
support  key  teaching  point(s). 

•  Replay  unmarked  portions  of  the  mission,  to  address 
unanticipated  points  or  questions. 

•  Use  audio  track  (radio  traffic)  during  AAR,  to  support  teaching 
points  or  answer  questions. 

Playback  Pause 

•  Temporarily  suspend  playback  during  the  AAR,  while  a  key 
teaching  point  is  discussed. 

First-Person  View 

•  Show  what  a  friendly  Soldier  was  seeing,  to  support  teaching 
points  or  answer  questions. 

•  Show  what  an  enemy  Soldier  was  seeing,  to  support  teaching 
points  or  answer  questions. 

•  Compare  the  views  of  two  or  more  Soldiers,  to  illustrate 
different  perspectives. 

Third-Person  View 

•  Show  what  an  observer  could  see  over  a  Soldier’s  shoulder,  to 
support  a  teaching  point. 

•  Compare  an  observer’s  view  for  two  or  more  Soldiers,  to 
illustrate  different  perspectives. 

Entity-Attached 

View 

•  Show  the  operation  from  the  perspective  of  a  vehicle  or 
dismount  (friendly  or  enemy). 

•  Highlight  key  actions  from  the  perspective  of  a  vehicle  or 
dismount  (friendly  or  enemy). 

•  Lock  onto  a  friendly  vehicle  or  dismount  to  show  its  actions 
during  a  critical  event. 

•  Compare  the  views  of  two  or  more  vehicles/dismounts,  to 
illustrate  different  perspectives. 

Cursor/Pointer 

•  Use  the  workstation  cursor  to  point  to  a  key  feature  of  a  static 
(e.g.,  map)  display. 

•  Use  the  cursor  to  highlight  key  feature(s)  of  dynamic  (playback) 
scenes  or  imagery. 

•  Use  the  cursor  to  indicate  objectives,  boundaries,  routes, 
obstacles,  etc.  on  a  static  display. 

Timestamp 

(Bookmark) 

•  Use  the  timestamps  of  selected  bookmarks  to  confirm  the 
chronological  order  of  events. 

•  Use  a  bookmark’s  timestamp  to  verify  timing  in  support  of  a 
teaching  point  or  question. 

C-ll 


Hit  Lines 

•  Display  hit  lines  to  identify  shooters  and  their  targets  during 
mission  playback. 

•  Use  clock-time  information  to  determine  the  chronological 
order  of  firing  events. 

•  Display  hit  lines  during  playback  to  see  who  was  responsible 
for  fratricide(s)  and  when. 

Save/Edit  File 

•  Save  portions  of  the  mission  at  the  end  of  the  AAR,  to  be 
reviewed  with  written  notes  later. 

•  Subsequently  produce  a  video  for  teaching  TTP,  by  editing  a 
recorded  file  and  saving  it. 

•  Convert  VBS2  files  to  video  format  for  exporting,  so  they  can 
be  played  on  other  systems. 
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Acronyms 


AAR 

After  Action  Review 

AO 

Area  of  Operations 

ARI 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences 

C2 

Command  and  Control 

CCTT 

Close  Combat  Tactical  Trainer 

FRAPS 

Video  Capture  Software 

HLA 

High  Level  Architecture 

IED 

Improvised  Explosive  Device 

LVC 

Live,  Virtual,  Constructive 

LZ 

Landing  Zone 

MOUT 

Military  Operations  on  Urbanized  Terrain 
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RTE 

TTP 

UAV 

VBS2 

VOIP 


Real  Time  Editor 

Tactics,  Techniques,  and  Procedures 
Unmanned  Aerial  Vehicle 
Virtual  Battlespace-2 
Voice-Over-Internet-Protocol 
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